Posts

Survey on intermediate goals in AI governance 2023-03-17T12:44:42.305Z
Technological developments that could increase risks from nuclear weapons: A shallow review 2023-02-09T15:41:54.858Z
Adding important nuances to "preserve option value" arguments 2023-01-08T09:30:56.985Z
Where are you donating this year, and why? (Open thread) 2022-11-23T12:26:47.411Z
Collection of work on 'Should you should focus on the EU if you're interested in AI governance for longtermist/x-risk reasons?' 2022-08-06T16:49:43.529Z
EA Infrastructure Fund: September–December 2021 grant recommendations 2022-07-12T15:24:31.256Z
Interested in EA/longtermist research careers? Here are my top recommended resources 2022-06-26T17:13:12.441Z
Don’t think, just apply! (usually) 2022-04-12T08:06:42.396Z
Nuclear risk research ideas: Summary & introduction 2022-04-08T11:17:09.884Z
Propose and vote on potential EA Wiki articles / tags [2022] 2022-04-08T07:09:40.662Z
8 possible high-level goals for work on nuclear risk 2022-03-29T06:30:52.532Z
Collection of definitions of "good judgement" 2022-03-14T14:14:14.457Z
Why and how to be excited about megaprojects 2022-01-24T13:34:26.322Z
EA Infrastructure Fund: May–August 2021 grant recommendations 2021-12-24T10:42:08.969Z
Database of orgs relevant to longtermist/x-risk work 2021-11-19T08:50:43.284Z
Competition for "Fortified Essays" on nuclear risk 2021-11-17T20:55:36.992Z
Some readings & notes on how to do high-quality, efficient research 2021-11-17T17:01:39.326Z
"Slower tech development" can be about ordering, gradualness, or distance from now 2021-11-14T20:58:04.899Z
How to decide which productivity coach to try? 2021-11-12T08:04:26.974Z
Things I often tell people about applying to EA Funds 2021-10-27T06:51:00.530Z
List of EA funding opportunities 2021-10-26T07:49:42.576Z
"Nuclear risk research, forecasting, & impact" [presentation] 2021-10-21T10:54:27.494Z
When and how should an online community space (e.g., Slack workspace) for a particular type/group of people be created? 2021-10-07T12:41:45.909Z
Event on Oct 9: Forecasting Nuclear Risk with Rethink Priorities' Michael Aird 2021-09-29T17:45:03.718Z
Independent impressions 2021-09-26T18:43:59.538Z
Improving EAs’ use of non-EA options for research training, credentials, testing fit, etc. 2021-09-11T13:52:15.738Z
Books and lecture series relevant to AI governance? 2021-07-18T15:54:32.894Z
Announcing the Nuclear Risk Forecasting Tournament 2021-06-16T16:12:39.249Z
Why EAs researching mainstream topics can be useful 2021-06-13T10:14:03.244Z
Overview of Rethink Priorities’ work on risks from nuclear weapons 2021-06-10T18:48:35.871Z
Final Report of the National Security Commission on Artificial Intelligence (NSCAI, 2021) 2021-06-01T08:19:15.901Z
Notes on Mochary's "The Great CEO Within" (2019) 2021-05-29T18:53:24.594Z
Intervention options for improving the EA-aligned research pipeline 2021-05-28T14:26:50.602Z
Reasons for and against posting on the EA Forum 2021-05-23T11:29:10.948Z
Goals we might have when taking actions to improve the EA-aligned research pipeline 2021-05-21T11:16:48.273Z
What's wrong with the EA-aligned research pipeline? 2021-05-14T18:38:19.139Z
Improving the EA-aligned research pipeline: Sequence introduction 2021-05-11T17:57:51.387Z
Thoughts on "A case against strong longtermism" (Masrani) 2021-05-03T14:22:11.541Z
Thoughts on “The Case for Strong Longtermism” (Greaves & MacAskill) 2021-05-02T18:00:32.482Z
My personal cruxes for focusing on existential risks / longtermism / anything other than just video games 2021-04-13T05:50:22.145Z
On the longtermist case for working on farmed animals [Uncertainties & research ideas] 2021-04-11T06:49:05.968Z
The Epistemic Challenge to Longtermism (Tarsney, 2020) 2021-04-04T03:09:10.087Z
New Top EA Causes for 2021? 2021-04-01T06:50:31.971Z
Notes on EA-related research, writing, testing fit, learning, and the Forum 2021-03-27T09:52:24.521Z
Notes on Henrich's "The WEIRDest People in the World" (2020) 2021-03-25T05:04:37.093Z
Notes on "Bioterror and Biowarfare" (2006) 2021-03-01T09:42:38.136Z
A ranked list of all EA-relevant (audio)books I've read 2021-02-17T10:18:59.900Z
Open thread: Get/give feedback on career plans 2021-02-12T07:35:03.092Z
Notes on "The Bomb: Presidents, Generals, and the Secret History of Nuclear War" (2020) 2021-02-06T11:10:08.290Z
Books on authoritarianism, Russia, China, NK, democratic backsliding, etc.? 2021-02-02T03:52:43.821Z

Comments

Comment by MichaelA on Database of orgs relevant to longtermist/x-risk work · 2023-03-25T08:38:28.847Z · EA · GW

Apart Research

A*PART is an independent ML safety research and research facilitation organization working for a future with a benevolent relationship to AI.

We run AISI, the Alignment Hackathons, and an AI safety research update series.

Comment by MichaelA on Database of orgs relevant to longtermist/x-risk work · 2023-03-25T08:37:17.752Z · EA · GW

Also the European Network for AI Safety (ENAIS)

TLDR; The European Network for AI Safety is a central point for connecting researchers and community organizers in Europe with opportunities and events happening in their vicinity. Sign up here to become a member of the network, and join our launch event on Wednesday, April 5th from 19:00-20:00 CET!
 

Comment by MichaelA on Survey on intermediate goals in AI governance · 2023-03-17T13:08:41.247Z · EA · GW

...and while I hopefully have your attention: My team is currently hiring for a Research Manager! If you might be interested in managing one or more researchers working on a diverse set of issues relevant to mitigating extreme risks from the development and deployment of AI, please check out the job ad!

The application form should take <2 hours. The deadline is the end of the day on March 21. The role is remote and we're able to hire in most countries.

People with a wide range of backgrounds could turn out to be the best fit for the role. As such, if you're interested, please don't rule yourself out due to thinking you're not qualified unless you at least read the job ad first!

Comment by MichaelA on Database of orgs relevant to longtermist/x-risk work · 2023-03-06T10:03:38.148Z · EA · GW

Riesgos Catastróficos Globales

Our mission is to conduct research and prioritize global catastrophic risks in the Spanish-speaking countries of the world. 

There is a growing interest in global catastrophic risk (GCR) research in English-speaking regions, yet this area remains neglected elsewhere. We want to address this deficit by identifying initiatives to enhance the public management of GCR in Spanish-speaking countries. In the short term, we will write reports about the initiatives we consider most promising. [Quote from Introducing the new Riesgos Catastróficos Globales team]

Comment by MichaelA on Database of orgs relevant to longtermist/x-risk work · 2023-03-06T10:01:38.534Z · EA · GW

Epoch

We’re a team of researchers investigating and forecasting the development of advanced AI.

Comment by MichaelA on Database of orgs relevant to longtermist/x-risk work · 2023-03-05T10:22:27.018Z · EA · GW

International Center for Future Generations

The International Center for Future Generations is a European think-and-do-tank for improving societal resilience in relation to exponential technologies and existential risks.

As of today, their website lists their priorities as:

  • Climate crisis
  • Technology [including AI] and democracy
  • Biosecurity
Comment by MichaelA on EV UK board statement on Owen's resignation · 2023-02-21T08:17:55.768Z · EA · GW

I appreciate you sharing this additional info and reflections, Julia. 

I notice you mention being friends with Owen, but, as far as I can tell, the post, your comment, and other comments don't highlight that Owen was on the board of (what's now called) EV UK when you learned about this incident and tried to figure out how to deal with it, and EV UK was the umbrella organization hosting the org (CEA) that was employing you (including specifically for this work).[1] This seems to me like a key potential conflict of interest, and like it may have warranted someone outside CEA being looped in to decide what to do about this incident. At first glance, I feel confused about this not having been mentioned in these comments. I'd be curious to hear whether you explicitly thought about that when you were thinking about this incident in 2021?

That is, if I understand correctly, in some sense Owen had a key position of authority in an organization that in turn technically had authority over the organization you worked at. That said, my rough impression from the outside is that, prior to November 2022, the umbrella organization in practice exerted little influence over what the organizations it hosted did. So this conflict of interest was probably in practice weaker than it would've looked on paper. But still it seems noteworthy.

More generally, this makes me realise that it seems like it would be valuable for the community health team to:

  • have a standard protocol for dealing with reports/incidents related to leadership or board members at CEA itself, EV UK, and EV US
    • And perhaps also to other staff at those orgs, and senior staff at any funder providing these orgs with (say) >10% of their funding (which I'd guess might just be Open Phil?)
  • have that protocol try to reduce reliance on the community health team's own judgment/actions in those cases
    • Probably meaning finding someone similarly suited to this kind of work but who sits outside of those lines of authority, who can deal with the small minority of cases that this protocol applies to. Or perhaps multiple people, each handling a different subset of cases.

(I'm not saying this should extend to the other orgs EV UK / EV US host, e.g. GWWC or 80k, just CEA and the umbrella orgs themselves.)

I'd be curious to hear whether such a thing is already in place, and if so what it looks like.

Caveats in a footnote. [2]

(I wrote this just in a personal capacity. I didn't run this by anyone.)

  1. ^

    I'n not sure if this terminology is exactly right. I'm drawing on the post CEA Disambiguation

  2. ^

    • I'm certainly not an expert on how these sorts of things should be handled.
    • I think your team has a tricky job that has to involve many tradeoffs.
    • I think it's probably disproportionately common for the times when your actions were followed by bad outcomes (even if that wasn't caused by your action, or was you making a good bet but getting unlucky) to become visible and salient.
    • I think there are likely many considerations I'm missing.
    • I didn't saliently notice worries or ideas about how should the community health team should handle various conflicts of interest prior to November 2022, and didn't saliently notice the question of what to do about incidents relating to senior staff at CEA / EV UK / EV US until this morning, and of course things tend to be easier to spot in hindsight. (OTOH I just hadn't spent much time thinking about the community health team at all, since it wasn't very relevant to my job.)
Comment by MichaelA on Database of orgs relevant to longtermist/x-risk work · 2023-02-14T12:13:03.267Z · EA · GW

Harvard AI Safety Team (HAIST), MIT AI Alignment (MAIA), and Cambridge Boston Alignment Initiative (CBAI)

These are three distinct but somewhat overlapping field-building initiatives. More info at Update on Harvard AI Safety Team and MIT AI Alignment and at the things that post links to.

Comment by MichaelA on Database of orgs relevant to longtermist/x-risk work · 2023-02-12T10:39:57.965Z · EA · GW

Labour for the Long Term

Is Britain prepared for the challenges ahead?
We face significant risks, from climate change to pandemics, to digital transformation and geopolitical tensions. We need social-democratic answers to create a fair and resilient future.

Our vision
A leading role for the UK
Many long-term issues have an important political dimension in which the UK can play a leading role. Building on the work of previous Labour governments, we see a future where the UK can play a larger role in areas such as in reducing international tensions and in becoming a world leader in green technology.

Comment by MichaelA on Database of orgs relevant to longtermist/x-risk work · 2023-02-12T10:38:00.044Z · EA · GW

Policy Foundry

an Australian-based organisations dedicated to developing high-quality and detailed policy proposals for the greatest challenges of the 21st century. [source]

Comment by MichaelA on Database of orgs relevant to longtermist/x-risk work · 2023-02-12T10:36:03.164Z · EA · GW

The Collective Intelligence Project

We are an incubator for new governance models for transformative technology.

Our goal: To overcome the transformative technology trilemma.

Existing tech governance approaches fall prey to the transformative technology trilemma. They assume significant trade-offs between progress, participation, and safety.

Market-forward builders tend to sacrifice safety for progress; risk-averse technocrats tend to sacrifice participation for safety; participation-centered democrats tend to sacrifice progress for participation.

Collective flourishing requires all three. We need CI R&D so we can simultaneously advance technological capabilities, prevent disproportionate risks, and enable individual and collective self-determination.

Comment by MichaelA on Technological developments that could increase risks from nuclear weapons: A shallow review · 2023-02-10T17:55:14.753Z · EA · GW

Just remembered that Artificial Intelligence and Nuclear Command, Control, & Communications: The Risks of Integration was written and published after I initially drafted this, so Will and I's post doesn't draw on  or reference this, but it's of course relevant too.

Comment by MichaelA on Technological developments that could increase risks from nuclear weapons: A shallow review · 2023-02-10T17:53:58.605Z · EA · GW

Just remembered that Artificial Intelligence and Nuclear Command, Control, & Communications: The Risks of Integration was written and published after I initially drafted this, so Will and I's post doesn't draw on  or reference this, but it's of course relevant too.

Comment by MichaelA on Donation recommendations for xrisk + ai safety · 2023-02-09T17:39:14.107Z · EA · GW

Glad to hear that!

Oh also, just noticed I forgot to add info on how to donate, in case you or others are interested - that info can be found at https://rethinkpriorities.org/donate 

Comment by MichaelA on Technological developments that could increase risks from nuclear weapons: A shallow review · 2023-02-09T17:35:29.900Z · EA · GW

An in-our-view interesting tangential point: It might decently often be the case that a technological development initially increases risks but then later increases risk by a smaller margin or even overall reduces risks. 

  • One reason this can happen is that developments may be especially risky in the period before states or other actors have had time to adjust their strategies, doctrine, procedures, etc. in light of the development.
  • Another possible reason is that a technology may be riskiest in the period when it is just useful enough to be deployed but not yet very reliable.
  • Geist and Lohn (2018) suggest this might happen, for the above two reasons, with respect to AI developments and nuclear risk:
    • “Workshop participants agreed that the riskiest periods will occur immediately after AI enables a new capability, such as tracking and targeting or decision support about escalation. During this break-in period, errors and misunderstandings are relatively likely. With time and increased technological progress, those risks would be expected to diminish. If the main enabling capabilities are developed during peacetime, then it may be reasonable to expect progress to continue beyond the point at which they could be initially fielded, allowing time for them to increase in reliability or for their limitations to become well understood. Eventually, the AI system would develop capabilities that, while fallible, would be less error-prone than their human alternatives and therefore be stabilizing in the long term”
Comment by MichaelA on Donation recommendations for xrisk + ai safety · 2023-02-08T12:46:49.954Z · EA · GW

Rethink Priorities' AI Governance & Strategy team (which I co-lead) has room for more funding. There's some info about our work and the work of RP's other x-risk-focused team* here and elsewhere in that post. One piece of public work by us so far is Understanding the diffusion of large language models: summary. We also have a lot of work that's unfortunately not public, either because it's still in progress or e.g. due to information hazards. I could share some more info via a DM if you want.

We also have yet to release a thorough public overview of the team, but we aim to do so in the coming months.

(*That other team - the General Longtermism team - may also be interested in funding, but I don't want to speak for them. I could probably connect you with them  if you want.)

Comment by MichaelA on Assessing China's importance as an AI superpower · 2023-02-05T06:56:58.623Z · EA · GW

Thanks - I only read this linkpost and Haydn's comment quoting your summary, not the linked post as a whole, but this seems to me like probably useful work.

One nitpick: 

It seems likely to me that the US is currently much more likely to create transformative AI before China, especially under short(ish) timelines (next 5-15 years) - 70%.

I feel like it'd be more useful/clearer to say "It seems x% likely that the US will create transformative AI before China, and y% likely if TAI is developed in short(ish) timelines (next 5-15 years)". Because:

  • At the moment, you're saying it's 70% likely that the US will be "much more likely", i.e. giving a likelihood of a qualitatively stated (hence kind-of vague) likelihood. 
  • And that claim itself seems to be kind-of but not exactly conditioned on short timelines worlds. Or maybe instead it's a 70% chance of the conjunction of "the US is much more likely (not conditioning on timelines)" and "this is especially so if there are short timelines". It's not really clear which one. 
    • And if it's the conjunction, that seems less useful than knowing what odds you assign to each of the two claims separately.
Comment by MichaelA on When reporting AI timelines, be clear who you're deferring to · 2023-02-05T06:00:35.024Z · EA · GW

Thanks, this seems right to me.

Are the survey results shareable yet? Do you have a sense of when they will be? 

Comment by MichaelA on Database of orgs relevant to longtermist/x-risk work · 2023-01-22T02:56:32.200Z · EA · GW

Also Cavendish Labs:

Cavendish Labs is a 501(c)(3) nonprofit research organization dedicated to solving the most important and neglected scientific problems of our age.

We're founding a research community in Cavendish, Vermont that's focused primarily on AI safety and pandemic prevention, although we’re interested in all avenues of effective research.

projectspapersprizes

Comment by MichaelA on CEA didn't want Zvi Moshowitz, Zeynep Tufekci, or Helen Chu in the Covid documentary they commissioned? · 2023-01-20T10:25:08.429Z · EA · GW

(I've now responded via email.)

Comment by MichaelA on CEA didn't want Zvi Moshowitz, Zeynep Tufekci, or Helen Chu in the Covid documentary they commissioned? · 2023-01-20T04:32:01.935Z · EA · GW

(I wrote this comment in a personal capacity, intending only to reflect my own views / knowledge.)

Hi,

In 2021, the EA Infrastructure Fund (which is not CEA, though both are supported and fiscally sponsored by Effective Ventures) made a grant for preparatory work toward potentially creating a COVID-related documentary.[1] I was the guest fund manager who recommended that grant. When I saw this post, I guessed the post was probably related to that grant and to things I said, and I’ve now confirmed that.

This post does not match my memory of what happened or what I intended to communicate, so I'll clarify a few things:

  • The EA Infrastructure Fund is not CEA, and I’m just one of its (unpaid, guest) fund managers. So what I said shouldn’t be treated as “CEA’s view”. 
  • The EAIF provided this grant in response to an application the grantseekers made, rather than “commissioning” it. 
  • When evaluating this grant, I consulted an advisor from the COVID forecasting space and another from the biosecurity space. They both flagged one of the people mentioned in the title of this post as seeming maybe unwise to highlight in this documentary. 
    • But I don’t recall this having been about that person having disagreed with authorities. 
      • Instead, my notes confirm that one of the advisors solely mentioned that that person’s work hadn’t been very impactful, since it was principally used just by the EA and rationality communities for their own benefit (rather than having impact at a larger scale). 
      • And my rough memory (but I lack clear notes in this case) is that the other advisor basically thought this person’s work had made overly strong claims overly confidently and that this didn’t seem like good epistemics to highlight. (As opposed to the worry being that the person was overly divisive or controversial.)
    • I passed a summarized version of those thoughts on to the grantseekers, along with various other bits of tentative feedback/advice.
    • I didn’t make the grant conditional on excluding anyone. And if I recall correctly, I didn’t feel very confident about whether and how to feature specific people in the documentary and I explicitly communicated that I was giving various bits of input merely as input and I want the grantees to make their own decisions. 
      • (Though this post makes me think that I failed to adequately emphasize that this was just input, and that therefore in this case it may have been better to not give the input at all. I now intend to adjust my future communications in light of that.) 
  • I don’t immediately recognize the other two names in the title of this post, and couldn’t find any comments from me about those people in my main grant evaluation notes doc or a few other related docs. So I don’t know why they’re mentioned in the title or screenshot. 

[1] The grant is described in one of EAIF’s public payout reports. But it doesn’t seem productive to name the grantees here. 

(EDIT: I wrote this and hit publish before seeing Rachel also commented shortly beforehand. Her comment does not match my memory of events in a few ways additional to what I noted in this comment. I might say more on that later, but I'd guess it's not very productive to discuss this further here. Regardless, as noted in my comment, it does seem to me that in this case I failed to adequately emphasize that my input was intended just as input, and I regret that.)

Comment by MichaelA on AGI and the EMH: markets are not expecting aligned or unaligned AI in the next 30 years · 2023-01-19T09:17:05.911Z · EA · GW

Minor (yet longwinded!) comment: FWIW, I think that:

  • Rohin's comment seems useful
  • Stephen's and your rebuttal also seem useful
  • Stephen's and your rebuttal does seem relevant to what Rohin said even with his caveat included, rather than replying to a strawman
  • But the phrasing of your latest comment[1] feels to me overconfident, or somewhat like it's aiming at rhetorical effect rather than just sharing data and inferences, or somewhat soldier-mindset-y
    •  In particular, personally I dislike the use of "110%", "maximally", and maybe "emphatically".
    • My intended vibe here isn't "how dare you" or "this is a huge deal". 
      • I'm not at all annoyed at you for writing that way, I (think I) can understand why you did (I think you're genuinely confident in your view and feel you already explained it, and want to indicate that?), and I think your tone in this comment is significant less important than your post itself. 
    • But I do want to convey that  I think debates and epistemics on the Forum will typically be better if people avoid adding such flourishes/absolutes/emphatic-ness in situations like this (e.g., where the writing shouldn't be optimized for engagingness or persuasion but rather collaborative truth-seeking, and where the disagreed-with position isn't just totally crazy/irrelevant). And I guess what I’d prefer pushing toward is a mindset of curiosity about what’s causing the disagreement and openness to one’s own view also shifting.

(I should flag that I didn't read the post very carefully, haven't read all the comments, and haven't formed a stable/confident view on this topic. Also I'm currently sleep-deprived and expect my reasoning isn't super clear unfortunately.)

  1. ^

    I also think the comment is overconfident in substance, but that's something that happens often in productive debates, and I think that cost is worth paying and hard to totally avoid if we want productive debates to happen.)

     

Comment by MichaelA on Propose and vote on potential EA Wiki articles / tags [2022] · 2023-01-06T09:17:11.155Z · EA · GW

(Update: I've now made this entry.)

Publication norms

I haven't checked how many relevant posts there are, but I'd guess 2-10 quite relevant and somewhat notable posts? 

Related entries

proliferation | AI governance | AI forecasting | [probably some other things]

Comment by MichaelA on Database of orgs relevant to longtermist/x-risk work · 2023-01-03T09:38:14.084Z · EA · GW

Also the Forecasting Research Institute

The Forecasting Research Institute (FRI) is a new organization focused on advancing the science of forecasting for the public good. 

[...] our team is pursuing a two-pronged strategy. One is foundational, aimed at filling in the gaps in the science of forecasting that represent critical barriers to some of the most important uses of forecasting—like how to handle low probability events, long-run and unobservable outcomes, or complex topics that cannot be captured in a single forecast. The other prong is translational, focused on adapting forecasting methods to practical purposes: increasing the decision-relevance of questions, using forecasting to map important disagreements, and identifying the contexts in which forecasting will be most useful.

[...] Our core team consists of Phil Tetlock, Michael Page, Josh Rosenberg, Ezra Karger, Tegan McCaslin, and Zachary Jacobs. We also work with various contractors and external collaborators in the forecasting space.

Comment by MichaelA on Database of orgs relevant to longtermist/x-risk work · 2023-01-03T09:36:36.455Z · EA · GW

Also School of Thinking

School of Thinking (SoT) is a media startup.

Our purpose is to spread Effective Altruist, longtermist, and rationalist values and ideas as much as possible to the general public by leveraging new media. We aim to reach our goal through the creation of high-quality material posted on an ecosystem of YouTube channels, profiles on social media platforms, podcasts, and SoT's website. 

Our priority is to produce content in English and Italian, but we will cover more languages down the line. We have been funded by the Effective Altruism Infrastructure Fund (EAIF) and the FTX Future Fund.

Comment by MichaelA on MichaelA's Shortform · 2023-01-03T08:46:58.576Z · EA · GW

Sometime after writing this, I saw Asya Bergal wrote an overlapping list of downsides here

"I do think projects interacting with policymakers have substantial room for downside, including:

  • Pushing policies that are harmful
  • Making key issues partisan
  • Creating an impression (among policymakers or the broader world) that people who care about the long-term future are offputting, unrealistic, incompetent, or otherwise undesirable to work with
  • “Taking up the space” such that future actors who want to make long-term future-focused asks are encouraged or expected to work through or coordinate with the existing project"
Comment by MichaelA on MichaelA's Shortform · 2023-01-03T08:43:06.739Z · EA · GW

Types of downside risks of longtermism-relevant policy, field-building, and comms work [quick notes]

I wrote this quickly, as part of a set of quickly written things I wanted to share with a few Cambridge Existential Risk Initiative fellows. This is mostly aggregating ideas that are already floating around. The doc version of this shortform is here, and I'll probably occasionally update that but not this.

"Here’s my quick list of what seem to me like the main downside risks of longtermism-relevant policy work, field-building (esp. in new areas), and large-scale communications.

  1. Locking in bad policies
  2. Information hazards (primarily attention hazards)
  3. Advancing some risky R&D areas (e.g., some AI hardware things, some biotech) via things other than infohazards
    • e.g., via providing better resources for upskilling in some areas, or via making some areas seem more exciting
  4. Polarizing / making partisan some important policies, ideas, or communities 
  5. Making a bad first impression in some communities / poisoning the well
  6. Causing some sticky yet suboptimal framings or memes to become prominent 
    • Ways they could be suboptimal: inaccurate, misleading, focusing attention on the wrong things, non-appealing
    • By “sticky” I mean that, one these framings/memes are prominent, it’s hard to change that
  7. Drawing more attention/players to some topics, and thereby making it less the case that we’re operating in a niche field and can have an outsized influence

Feel free to let me know if you’re not sure what I mean by any of these or if you think you and me chatting more about these things seems worthwhile. 

Also bear in mind the unilateralist's curse

None of this means people shouldn’t do policy stuff or large-scale communications. Definitely some policy stuff should happen already, and over time more should happen. These are just things to be aware of so you can avoid doing bad things and so you can tweak net positive things to be even more net positive by patching the downsides.

See also Hard-to-reverse decisions destroy option value  and Adding important nuances to "preserve option value" arguments"

Comment by MichaelA on Propose and vote on potential EA Wiki articles / tags [2022] · 2023-01-03T08:22:32.952Z · EA · GW

EU AI Act and/or NIST AI Risk Management Framework

These are quite separate, but I mention them together because they're both specific pieces of upcoming AI policy that I think many experts think are pretty important. It's pretty unclear to me whether we should have entries for these two specific things and for things like this in general. 

  • There are several posts focused on or touching on each of these things, and it seems nice to have a way to collect them. 
  • But maybe if we had entries for each piece of policy that's roughly this important, according to each major EA cause area, that'd be dozens and would be too many?
Comment by MichaelA on Main paths to impact in EU AI Policy · 2023-01-03T08:13:36.013Z · EA · GW

Thanks for this post. Readers who found this useful would probably also find some things linked to from my post Collection of work on 'Should you should focus on the EU if you're interested in AI governance for longtermist/x-risk reasons?' useful, since those posts tackle a similar topic but with different framings / different specific points.

(I've also now added this post to that collection.)

Comment by MichaelA on MichaelA's Shortform · 2023-01-02T14:07:28.652Z · EA · GW

I've now turned this into a top-level post, and anyone who wants to read this should now read that version rather than this shortform.

Adding important nuances to "preserve option value" arguments 

Summary

I fairly commonly hear (and make) arguments like "This action would be irreversible. And if we don't take the action now, we can still do so later. So, to preserve option value, we shouldn't take that action, even if it would be good to do the action now if now was our only chance."[1]  

This is relevant to actions such as:

  • doing field-building to a new target audience for some important cause area
  • publicly discussing of some important issue in cases where that discussion could involve infohazards, cause polarization, or make our community seem wacky

I think this sort of argument is often getting at something important, but in my experience such arguments are usually oversimplified in some important ways. This shortform is a quickly written[2]  attempt to provide a more nuanced picture of that kind of argument. My key points are:

  1. "(Ir)reversibility" is a matter of degree (not a binary), and a matter of the expected extent to which the counterfactual effects we're considering causing would (a) fade by default if we stop fuelling them, and/or (b) could be reversed by us if we actively tried to reverse them. 
    1. Sometimes we may be surprised to find that something does seem decently reversible.
  2. The "option value" we retain is also a matter of degree, and we should bear in mind that delays often gradually reduce total benefits and sometimes mean missing key windows of opportunity. 
  3. Delaying can only be better than acting now if we expect we'll be able to make a better-informed decision later and/or we expect the action to become more net-positive later. 
    1. If we don't expect our knowledge will improve in relevant ways nor the act will become more valuable/less harmful, or we expect minor improvements that are outweighed by the downsides or delay, we should probably just act now if the action does seem good. 

But again, I still think "option value" arguments are often getting at something important; I just think we may often make better decisions if we also consider the above three nuances when making "option value" arguments. And, to be clear, I definitely still think it's often worth avoiding, delaying, or consulting people about risky-seeming actions rather than just taking them right now. 

I'd welcome feedback on these ideas. Also please let me know if you think this should be a top-level post. 

1. On "irreversibility"

In some sense, all actions are themselves irreversible - if you do that action, you can never make it literally the case that you didn't do that action. But, of course, that doesn't matter. The important question is instead something like "If we cause this variable to move from x to y, to what extent would our counterfactual impact remain even if we later start to wish we hadn't had that impact and we adjust our behaviors accordingly?" E.g., if we make a given issue something that's known by and salient to a lot of politicians and policymakers, to what extent, in expectation, will that continue to be true even if we later realise we wish it wasn't true? 

And this is really a question of degree, not a binary. 

There are two key reasons why something may be fairly reversible:

  • Our counterfactual effects may naturally wash out
    • The variable may gradually drift back to the setting it was at before our intervention
    • Or it may remain at the setting we put it to, but with it becoming increasingly likely over time that that would've happened even in the absence of our intervention, such that our counterfactual impact declines
    • For example, let's say we raise the salience of some issue to politicians and policymakers because it seems ~60% likely that that's a good idea, ~20% likely it's ~neutral, and ~20% likely it's a bad idea. Then we later determine it seems it was a bad idea after all, so we stop taking any actions to keep salience high. In that case:
      • The issue may gradually fall off these people's radars again, as other priorities force themselves higher up the agenda
      • Even if the issue remains salient or increases in salience, it could be that this or some fraction of it would've happened anyway, just on a delay
        • This is likely for issues that gradually become obviously real and important and where we notice the issues sooner than other key communities do
        • We could imagine a graph with one line showing how salience of the issue would've risen by default without us, another line showing how salience rises earlier or higher if we make that happen, and a third line for if we take the action but then stop. That third line would start the same as the "we make that happen" line, then gradually revert toward the "what would've happened by default" line.
  • We may be able to actively (partially) reverse our effects
    • I expect this effect would usually be less important than the "naturally wash out" effect.
      • Basically because when I tried to think of some examples, they all seemed either hard to achieve big results from or like they'd require "weird" or "common sense bad" actions like misleading people. 
    • But perhaps sometimes decently large effects could be achieved from this? 
    • For example, we could try to actively reduce the salience of an issue we previously increased the salience of, such as by contacting the people who we convinced and who most started to increase the issue's salience themselves (e.g., academics who started publishing relevant papers), and explaining to them our reasoning for now thinking it's counterproductive to make this issue more salient.

2. On "we can still do it later"

In some sense, it's always the case that if you don't take an action at a given time, you can't later do exactly that same action or achieve exactly the same effects anymore. Sometimes this hardly matters, but sometimes it's important. The important question is something like "If we don't take this action now, to what extent could we still achieve similar expected benefits with similarly low expected harms via taking a similar action later on?" 

I think very often significant value is lost by delaying net-positive actions. E.g., in general and all other factors held constant:

  • delaying field-building will reduce the number of full-time-equivalent years spent on key issues before it's "too late anyway" (e.g., because an existential catastrophe has happened or the problem has already been solved)
  • delaying efforts to improve prioritization & understanding of some issue will reduce the number of "policy windows" that occur between those efforts & the time when it's too late anyway 

I also think that sometimes delay could mean we miss a "window of opportunity" for taking an action with a similar type and balance of benefits to harms of the action we have in mind. That is, there may not just be a decay in the benefits, but rather a somewhat "qualitative" shift in whether "something like this action" is even on the table. For example, we may miss the one key policy window we were aiming to affect.

(Somewhat relevant: Crucial questions about optimal timing of work and donations.)

3. Will we plausibly have more reason to do it later than we do now? 

Delaying can only be better than acting now if at least one of the following is true:

  • We expect we'll be able to make a better-informed decision later
    • e.g., because our relevant knowledge will improve
  • We expect the action to become more net-positive later
    • e.g., because we expect favorable changes in background variables - the time will become "more ripe"

The more we expect those effects, the stronger the case for delay. The less we expect those effects, the weaker the case for delay. (A simplified way of saying this is "Why bother delaying your decision if you'd just later be facing the same or worse decision with the same or worse info?")

This can be weighed up against the degree to which we should worry about irreversibility and the degree to which we should worry about the costs of delay, in order to decide whether to act now. (Assuming the act does seem net positive & worth prioritizing, according to our current all-things-considered best guess.)

I think it's usually true that we'll (in expectation) be able to make a better-informed decision later, but how true that is can vary a lot between cases, and that magnitude matters if there are costs to delay.

I think it's sometimes true that the action will become more net-positive later, but probably usually the opposite is true (as discussed in the prior section).  

  1. ^

    See for example this post: Hard-to-reverse decisions destroy option value
     

    I read that post ~4 years ago and remember thinking it made good points and is valuable. I expect if I re-read it I'd still agree with it. I don't think I'd explicitly noticed the nuances this shortform expresses when I read that post, and I didn't check today whether that post already accounts for these nuances well. 

  2. ^

    I expect that some of my points are obvious and that some readers might find it arrogant/naive/weird that I wrote this without citing x y z literatures. It also seems plausible some of my points or uses of terminology are mistaken. Please feel free to mention relevant literatures and feel encouraged to highlight potential mistakes!

Comment by MichaelA on A list of EA-related podcasts · 2022-12-31T13:26:35.411Z · EA · GW

I also recommend Joe Carlsmith Audio.

Audio versions of essays by Joe Carlsmith. Philosophy, futurism, and other topics. Text versions at joecarlsmith.com.

Joe reads the essays himself.

Comment by MichaelA on A list of EA-related podcasts · 2022-12-31T13:25:15.180Z · EA · GW

Also the Future Matters Reader:

Future Matters Reader uses text-to-speech software to convert into audio the writings summarized in the Future Matters newsletter.

It seems like a lot of those writings weren't on Nonlinear's podcast feeds, either due to not being on the EA Forum / LessWrong / Alignment Forum or for some other reasons. 

Comment by MichaelA on Future Matters #6: FTX collapse, value lock-in, and counterarguments to AI x-risk · 2022-12-31T13:24:28.627Z · EA · GW

Thanks for your work on this! I find this newsletter useful, and also appreciate you making the Future Matters Reader feed for audio versions of many of the writings covered in the newsletter. It seems like a lot of those writings weren't on Nonlinear's podcast feeds, either due to not being on the EA Forum / LessWrong / Alignment Forum or for some other reasons, so this seems useful and I've now downloaded a bunch of things in your feed.

(I'm leaving this comment partly to make other readers aware of this.)

Comment by MichaelA on MichaelA's Shortform · 2022-12-29T19:11:42.250Z · EA · GW

Often proceed gradually toward soliciting forecasts and/or doing expert surveys

tl;dr: I think it's often good to have a pipeline from untargeted thinking/discussion that stumbles upon important topics, to targeted thinking/discussion of a given important topic, to expert interviews on that topic, to soliciting quantitive forecasts / doing large expert surveys.

I wrote this quickly. I think the core ideas are useful but I imagine they're already familiar to e.g. many people with experience making surveys.[1] I'm not personally aware of an existing writeup on this and didn't bother searching for one, but please comment if you know of one!

Introduction

Let's say you wanna get a better understanding of something. If you know exactly and in detail what it is that you want to get a better understanding of, two tools that can be very useful are forecasts and expert surveys. More specifically, it can be very useful to generate well-operationalized quantitative or fixed-choice questions and then get those questions answered by a large number of people with relevant expertise and/or good forecasting track records. 

But it's probably best to see that as an end point, rather than jumping to it too soon, for two reasons:

  • Getting responses is costly
    • Getting a lot of people with relevant expertise and/or good forecasting track records to answer your questions probably requires significant effort or money. 
    • It may also involve substantial opportunity cost, if those people are working on important, net-positive things.
  • Generating questions is hard
    • I've both done and observed a decent amount of forecasting question writing and survey design. It seems to me that it's harder to actually do well than most people would probably expect, and that people often don't realise they haven't done it well until after they get some feedback or some answers.
    • One difficulty is having even a rough sense of what it's best to ask about. 
    • Another difficulty is figuring out what precisely what to ask about and phrasing that very clearly, such that respondents can easily understand your question, they interpret it how you wanted, and the question covers and captures all of the relevant & useful thoughts they have to share.
      • This often requires/warrants a lot of thought, multiple rounds of feedback, and multiple rounds of testing on an initial batch of respondents.

So if you jump to making forecasting questions or surveys too early, you may:

  • waste a lot of your or other people's time/money on unimportant topics/questions
  • get responses that are confusing or misleading since the question phrasings were unclear
  • fail to hear a lot of the most interesting things people had to share, since you didn't ask about those things and your questions had precise scopes

...especially because forecasting questions and surveys are typically "launched" to lots of people at once, so you may not be able to or think to adapt your questions/approach in light of the first few responses, even if the first few give you reason to do so. 

The pipeline I propose for mitigating those issues

(Note: The boundaries between these "steps" are fuzzy. It probably often makes sense to jump back and forth to some extent. It probably also often makes sense to be at different stages at the same time for different subtopics/questions within a broad topic.)

  1. Untargeted thinking/discussion
    1. I.e., thinking/discussions/writing/research/whatever that either roams through many topics, or is fairly focused but not focused on the topic that this instance of "the pipeline" will end up focused on
    2. Sometimes this stumbles upon a new (to you) topic, or seems to suggest an already-noticed topic seems worth prioritizing further thought on
    3. Advantage: Very unconstrained; could stumble upon many things that you haven't already realised are worth prioritizing.
  2. Targeted thinking/discussion of a given important-seeming topic
    1. This is still unconstrained in its precise focus or its method, but now constrained to a particular broad topic.
    2. Advantage: Can go deeper on that topic, while still retaining flexibility regarding what the best scope, most important subquestions, etc. is
  3. Expert interviews on that topic
    1. Similar to the above "step", but now with a clearer sense of what questions you're asking and with more active effort to talk to experts specifically.
    2. Within this step, you might want to move through a pipeline with a similar rationale to the overall pipeline, moving from (a) talking in a fairly unstructured way to people with only moderate expertise and opportunity cost to (b) following a specified and carefully considered interview protocol in interviews with the very best experts to talk to on this topic.
      1. (b) could even essentially be a survey delivered verbally but with occasional unplanned follow-up questions based on what respondents said.
    3. Advantage: Get well-founded thoughts on well-considered questions with relatively low cost to these people's fairly scarce time.
  4. Soliciting quantitative forecasts and/or running expert surveys
    1. It may often be worth doing both.
      1. Probably often with some but not complete overlap in the questions and participants.
        1. Some questions are better suited to people with strong forecasting track records and others better suited to people with relevant expertise. 
    2. Within this step, you might want to undergo a pipeline with a similar rationale to the overall pipeline, with multiple waves of forecasting-soliciting / surveying that each have more questions, more precise operationalizations of questions, and/or more respondents.
    3. Advantage: Get a large volume of well-founded, easily interpretable thoughts on well-considered questions, with relatively low cost to each person's fairly scarce time (even if high cost in aggregate). 

Misc thoughts

  • For similar reasons, I think it's probably usually good for interview protocols and surveys to include at least one "Any other thoughts?" type question and perhaps multiple (e.g., one after each set of questions on a similar theme). 
  • Also for similar reasons, I think it's probably usually good to allow/encourage forecasters' to share whatever thoughts they have that they think are worth sharing, rather than solely soliciting their forecasts on the questions asked.
  1. ^

    The specific trigger for me writing this was that I mentioned the core idea of this shortform to a colleague it was relevant to, and they said it seemed useful to them. 

    Another reason I bothered to write it is that in my experience this basic idea has seemed valid and useful, and I think it would've been a little useful for me to have read this a couple years ago. 

Comment by MichaelA on EAGx application community norms we'd like to see · 2022-12-24T11:50:41.517Z · EA · GW

I agree that this post seems useful, and also that changing the title to something closer to the TL;DR would probably be good. Though maybe something more like "Please register for EAG(x)'s early & cancel ASAP if you can't attend"? (Since it seems like registering early is also a key part of the suggestion.) 

I do think the other norms in the final section seem noteworthy, but still overall it's probably a little better for the title to get the two most important points across.

Another small argument for changing the title is that the current title sounds like how an org / group who very regularly run EAG(x)'s would title a post, and so made me feel like there was a ~2/3 chance this post was written on behalf of CEA - even though I also saw the author names, and know that at least as of November you two didn't work for CEA. I think that effect would be slightly reduced by a title like what Holly or I proposed.

Comment by MichaelA on List of EA funding opportunities · 2022-12-09T23:56:55.156Z · EA · GW

Open Phil is seeking applications from grantees impacted by recent events

First part of the post:

"We (Open Phil) are seeking applications from grantees affected by the recent collapse of the FTX Future Fund (FTXFF) who fall within our long-termist focus areas (biosecurity, AI risk, and building the long-termist EA community). If you fit the following description, please fill out this application form.

We’re open to applications from:

  • Grantees who never received some or all of their committed funds from FTXFF.
  • Grantees who received funds, but want to set them aside to return to creditors or depositors.
    • We think there could be a number of complex considerations here, and we don’t yet have a clear picture of how we’ll treat requests like these. We’d encourage people to apply if in doubt, but to avoid making assumptions about whether you’ll be funded (and about what our take will end up being on what the right thing to do is for your case). (Additionally, we’re unsure if there will be legal barriers to returning funds.) That said, we’ll do our best to respond to urgent requests quickly, so you have clarity as soon as possible.
  • Grantees whose funding was otherwise affected by recent events.[1]"
     
Comment by MichaelA on List of EA funding opportunities · 2022-12-09T23:56:47.688Z · EA · GW

SFF Speculation Grants as an expedited funding source

Comment by MichaelA on List of EA funding opportunities · 2022-12-09T23:56:24.249Z · EA · GW

SFF Speculation Grants as an expedited funding source

"Hi everyone, SFF has received numerous emails recently from organizations interested in expedited funding.  I believe a number of people here already know about SFF Speculation Grants, but since we've never actually announced our existence on the EA Forum before:

The Survival and Flourishing Fund has a means of expediting funding requests at any time of year, via applications to our Speculation Grants program:

https://survivalandflourishing.fund/speculation-grants

SFF Speculation Grants are expedited grants organized by SFF outside of our biannual grant-recommendation process (the S-process). “Speculation Grantors” are volunteers with budgets to make these grants. Each Speculation Grantor’s budget grows or increases with the settlement of budget adjustments that we call “impact futures” (explained further below). Currently, we have a total of ~20 Speculation Grantors, with a combined budget of approximately $10MM (up from $4MM initially). Our process and software infrastructure for funding these grants were co-designed by Andrew Critch and Oliver Habryka.

For instructions on how to apply, please visit the link above.

For general information about the Survival and Flourishing Fund, see:

https://survivalandflourishing.fund/"

Comment by MichaelA on Where are you donating this year, and why? (Open thread) · 2022-11-23T15:33:55.281Z · EA · GW

I hadn't donated for the last couple years despite having taken the GWWC pledge. This is because:

  • It started to seem like I have sufficiently good judgement, domain knowledge, and connectedness* that my money would be most useful for making opportunities I have special knowledge of happen quickly, rather than donating to large existing opportunities
  • I didn't happen to stumble upon anything especially good
  • I kind-of procrastinated / got busy (without reducing willingness to give; I still planned to "catch up" on this giving)

The downfall of FTX and resignation of the Future Fund team make me think now is a good time for me to catch up on this backlog of giving, and maybe also do some of my coming years' giving at the same time. So I've reached out to ~6 (former) FTX regrantors I know to let them know I'm potentially open to donating to things they think are especially promising or urgent. 

*My evidence for this included being offered a role on the EA Infrastructure Fund, among other things.

Comment by MichaelA on Database of orgs relevant to longtermist/x-risk work · 2022-11-14T02:20:41.690Z · EA · GW

Also Research in Effective Altruism and Political Science (REAPS)

Comment by MichaelA on [deleted post] 2022-11-09T12:19:14.824Z

I'm not totally sure whether this should exist, and whether it should be called this.

Comment by MichaelA on [deleted post] 2022-11-09T12:18:58.149Z

I'm not totally sure whether this should exist, and whether it should be called this.

Comment by MichaelA on AI timelines by bio anchors: the debate in one place · 2022-11-07T17:00:14.195Z · EA · GW

Also this recent FLI podcast interview with Ajeya: Ajeya Cotra on Forecasting Transformative Artificial Intelligence
 

Comment by MichaelA on [deleted post] 2022-11-07T07:56:27.850Z

I think an ideal version of this page would (a) think more carefully about how to define things, and include links to references that informed the definition or show that it's fairly standard, and (b) add a section explaining why proliferation/diffusion matters to nuclear risk, AI, biorisk, and maybe other things, and perhaps also explaining some similarities and differences in how it works/matters in each of those areas.

Comment by MichaelA on [deleted post] 2022-11-07T07:54:36.799Z

It seems plausible the key term used here should be diffusion, technology diffusion, technology proliferation, or (if we want to focus this more) AI proliferation / AI diffusion. 

A downside of "proliferation" is that it has negative connotations and is associated with WMDs, which I think is actually fairly appropriate for AI and biotech but also does perhaps undersell their upside potential and could be seen as unfairly "skewing the argument" against openness with AI or biotech development.

A downside of "diffusion" is that I think a lot of insights and ideas and debates from nuclear/WMD (non)proliferation do transfer to AI and biotech. Another is that a quick google suggested "technology diffusion" mostly refers to a tech being gradually taken up and used by a large segment of a population, whereas I think really what's more important here is expanding a tiny circle of people who can use the very cutting edge techs to a small and then moderate circle, and what happens if one of them uses it badly, which then seems more like what's usually discussed under the term "proliferation". 

Comment by MichaelA on Propose and vote on potential EA Wiki articles / tags [2022] · 2022-10-29T17:08:32.427Z · EA · GW

[Update: I've now created this entry.]

benchmarks or AI benchmarks or something like that

Relevant posts include:

Related entries

https://forum.effectivealtruism.org/topics/ai-safety 
 

Comment by MichaelA on New Forum License: Creative Commons · 2022-10-27T08:35:58.479Z · EA · GW

Cool!

I'd like all my prior posts to have this license (unless they're linkposts to things I didn't write, I guess), but don't want to take the time to manually add that to all my posts. Could the Forum team do some fancy code magic such that then you can quickly apply add license statement at the bottom of all posts by a given author if they opt in to that?

Comment by MichaelA on List of EA funding opportunities · 2022-10-23T09:35:10.586Z · EA · GW

Another opportunity: Amplify creative grants

Some info from that post:

We're announcing a small grants program for creative media, supported by Hear This Idea and the FTX Future Fund regranting program.

[...]

Over the next few months, we’re aiming to make grants to podcasts and other creative media projects that spread ideas to help humanity navigate this century.

We’re interested in applications that look to cover topics related to (i) reducing existential risk, (ii) helping fix pressing global problems, and (iii) putting humanity on a positive long-term trajectory. More details on all three below.

We want to amplify ideas associated with effective altruism and longtermism, but you don’t need to be actively engaged with effective altruism to apply.

We’re excited to support projects (new and existing) in English and other languages — sharing important ideas with new, global, audiences.

If you are unsure whether your idea fits among the areas we outline below, please lean toward applying— we want to hear your ideas! The form is a single page of questions, and should easily take less than an hour. The first batch of grants will be approved around three weeks after this announcement. After that, we’ll approve grants on a rolling basis. Let us know if you have a deadline you need to hear back by, and we’ll do our best to accommodate and get back very quickly if necessary. If you have any questions which aren’t answered below, please contact us on grants@hearthisidea.com.

You can apply now by following this link.

[...]

October 22nd update: we've been really impressed with the number and quality of applications over the last few weeks; enough to disburse all of our initial pot to the (successful) applications we have already received. As such, we will no longer be actively considering new applications for the first set of grants. However, we will keep the application form open as a way to express interest in case we renew the program. Thanks for your understanding!

Comment by MichaelA on Be careful with (outsourcing) hiring · 2022-10-18T15:26:01.134Z · EA · GW

Oh crap, my bad, should've looked closer! (And that's ironic given my comment, although at least I'd say the epistemic standards for comments can/should be lower than for posts.) Sorry about that. 

Though still I think "not predictive" seems like a strong misrepresentation of that source. I think correlations of .41 and .31 would typically be regarded as moderate/weak and definitely not "approximately zero". (And then also the apparently high incremental validity is interesting, though not as immediately easy to figure out the implications of.)

I agree that "the points of [your] post rely only lightly on this article". But I still think this is a sufficiently strong & (in my view, seemingly) obvious* misrepresentation of the source that it would make sense to see this issue as a reason for readers to be skeptical of any other parts of the post that they haven't closely checked. 

(I also find it surprising that in your two replies in this thread you didn't note that the table indeed seems inconsistent with "not predictive".)

*I.e., it doesn't seem like it requires a very close/careful reading of the source to determine that it's saying something very different to "not predictive". (Though I may be wrong - I only jumped to the table, and I guess it's possible other parts of the paper are very misleadingly written.)

Comment by MichaelA on Be careful with (outsourcing) hiring · 2022-10-18T07:48:18.874Z · EA · GW

There's a decent amount in this post that I agree with, but also sufficiently poor epistemics/reasoning transparency[1] (in my view) and enough advice that seems inaccurate[2] that I would recommend people either (a) don't read this post or (b) read it only with substantial skepticism and alongside other sources of advice (and not just Manager Tools, though maybe use that as well, just with skepticism). One place other advice can be found is in this quick collection I made: Quick notes/links on hiring/vetting

I do think this post has some advice many people would benefit from, but the post also confidently asserts some things that I think are probably inaccurate or bad advice.[2] And (in my view) the post doesn't make it easy for people to discern the good from bad advice unless they already have good knowledge here (in which case they'll also gain little from the post), since the post provides very little sources or arguments and has the same confident tone for all its advice (rather than presenting some things as more tentative). 

Overall, I think that this post is below the epistemic standards we should want for the Forum. (I'm aware this is somewhat rude/aggressive to write, and I promise I come across as friendlier in person! Also, in part [1] of this comment I provide some concrete recommendations / constructive advice that may be helpful for future posts. And to be clear, I do expect that the author was well-intentioned and trying to be helpful when writing this.)


[1] Regarding poor epistemics / reasoning transparency in my view: It sounds like the author deliberately chose to release a short-ish and maybe quickly-ish written post rather than no post at all ("I won't be arguing every point fully. [...] That's because doing otherwise would have made the article ten times longer"). That's a reasonable choice. But in this case I think it turns out too much of this advice is probably bad for the resulting quick post to be useful to many people. I think the ideal version of this post would involve maybe a bit more word count and a bit longer time spent (but doesn't have to be much more) and would:

  • Indicate differing levels of confidence in different claims
  • Probably be substantially less confident on average
    • I'm not saying that all posts should be low-confidence. But I think that should probably be the case when:
      • the author has limited experience
        • I'm basing this on the author saying "Hiring is a wide field and I've only tilled a small patch of it myself". To be clear, I'm not saying people should only write posts about things they have lots of experience on, just that limited experience plus the following things should warrant lower confidence.
      • The author didn't have time to properly find sources or explain the arguments
        • This is relevant because in the process of doing so one often realises oneself that the claims rest on shakier ground than one thought.
      • The main source seems to be a podcast/site (Manager Tools) which (I think) itself has poor epistemics
        • I've engaged a fair bit with Manager Tools, and think it's useful overall, but in my experience they also usually provide no evidence or arguments for their claims except their personal experience, and I think many of their claims/advice are bad. So when I'm engaging with it I'm doing so with skepticism and sifting for a few takeaways that are in hindsight obviously true or that I can test out easily myself. 
  • Provide more of the reasoning/arguments for various claims, even if that just means saying more of what one already believes or thinks rather than thinking things through in more detail or finding more sources
  • Be based on more careful reading of sources, to avoid drawing from one source a conclusion that seems exactly opposite to what the source finds
    • (I haven't checked any of the other claims/sources properly, so this may not be the only such issue)
  • Probably defer less to sources which don't themselves have strong evidence bases or reasoning transparency (Manager Tools)
  • Maybe find more high-quality sources (though it'd also be reasonable to not bother with that)

[2] Some things in this post that I think are inaccurate or bad advice (though unfortunately don't have time to explain my reasoning for most of these):

  • The claim that personality tests aren't predictive of job performance
  • Saying it's bad to outsource things you don't know because you can't assess performance well in those cases, without flagging that this might not apply anywhere near as much in cases where an outsourcee comes highly recommended by people you trust and whose situation is similar to yours, and when the outsourcee has credible signals of sharing a lot of your values 
    • (E.g., if they're a credibly EA-aligned hiring service/advisor who comes recommended from relevant other people based on performance there.)
  • Suggesting filtering only via resumes rather than instead/also via screening questions that are basically a very short work test
  • Suggesting giving many interviews and relying heavily on them, and suggesting candidates answering the same questions from each interviewee
    • I think that latter advice could only make sense if interviewees don't have a pre-set rubric and are just assessing answers subjectively, which seems unwise to me, or if the interviews are close to unstructured rather than structured, which also seems unwise to me. Also, relative to work tests, this takes up a lot of candidate and esp. hirer time.
  • Making final decisions based on a meeting where people verbally share their assessments one at a time (which could then be anchored by whoever speaks first or groupthink or whatever), rather than collecting scores along the way and sticking to them by default (though with room to manoeuvre) to minimise noise (see also the book Noise)

Those last three things basically seem consistent with the ways traditional hiring is bad and seems based on prizing subjective judgement and not really quantitatively assessing things or checking what research shows works best, in contrast to what's common in EA orgs + leading tech firms (as far as I'm aware). And that seems consistent with my experience of the kind of advice Manager Tools tends to give.

Also, these are just most of the things that stood out to me when casually listening to the post (in audio form), and that's enough to make me think that there may also be some other issues.