Hi, I'm Holden Karnofsky. AMA about jobs at Open Philanthropy

post by HoldenKarnofsky · 2018-03-26T16:32:28.313Z · score: 18 (20 votes) · EA · GW · Legacy · 120 comments

Hi EA Forum,

I'm Holden Karnofsky I'm here to answer any questions about jobs at Open Philanthropy. I'll be here today from 9:30am to 12:30pm Pacific time  (with some breaks) and will likely respond to comments later on as well. 

We'd hate to miss out on strong applicants because of misconceptions about the roles, so I hope people will ask whatever is on their mind, on topics from office environment to day-to-day work to the likely long-term trajectory of the role. I think Open Philanthropy jobs are among the best possible ways for effective altruists to have impact, and I hope anyone who could imagine performing well in these jobs will at least consider applying!

Please post different questions as separate comments, for discussion threading.

Looking forward to it!

Added 12:32pm Pacific time: This concludes the "official" portion of the AMA, but feel free to post more questions; we may respond to them later on!

120 comments

Comments sorted by top scores.

comment by itty · 2018-03-26T18:10:11.833Z · score: 9 (9 votes) · EA(p) · GW(p)

What are things that previous Research Analysts have struggled with at Open Phil? What are reasons others have found it not to be a good fit?

comment by HoldenKarnofsky · 2018-03-26T18:37:07.286Z · score: 7 (6 votes) · EA(p) · GW(p)

A few things that come to mind:

  1. The work is challenging, and not everyone is able to perform at a high enough level to see the career progression they want.

  2. The culture tends toward direct communication. People are expected to be open with criticism, both of people they manage and of people who manage them. This can be uncomfortable for some people (though we try hard to create a supportive and constructive context).

  3. The work is often solitary, consisting of reading/writing/analysis and one-on-one checkins rather than large-group collaboration. It's possible that this will change for some roles in the future (e.g. it's possible that we'll want more large-group collaboration as our cause prioritization team grows), but we're not sure of that.

comment by HoldenKarnofsky · 2018-03-26T19:41:28.176Z · score: 8 (8 votes) · EA(p) · GW(p)

This is still a common practice. The point of it isn't to evaluate employees by # of hours worked; the point is for their manager to have a good understanding of how time is being used, so they can make suggestions about what to go deeper on, what to skip, how to reprioritize tasks, etc.

Several employees simply opt out from this because they prefer not to do it. It's an optional practice for the benefit of employees rather than a required practice used for performance assessment.

comment by Ben Pace · 2018-03-26T18:24:01.052Z · score: 8 (8 votes) · EA(p) · GW(p)

I’m pretty confused about the work of the RA role - it seems to include everything from epidemiological literature reviews to philosophical work on population ethics to following up on individual organisations you’ve funded.

Could you give some concrete info about how you and the RA determine what the RA works on?

comment by HoldenKarnofsky · 2018-03-26T18:58:19.292Z · score: 7 (7 votes) · EA(p) · GW(p)

The role does include all three of those things, and I think all three things are well served by the job qualifications listed in the posting. A common thread is that all involve trying to deliver an informative, well-calibrated answer to an action-relevant question, largely via discussion with knowledgeable parties and critical assessment of evidence and arguments.

In general, we have a list of the projects that we consider most important to complete, and we look for good matches between high-ranked projects and employees who seem well suited to them. I expect that most entry-level Research Analysts will try their hand at both cause prioritization and grant investigation work, and we'll develop a picture of what they're best at that we can then use to assign them more of one or the other (or something else, such as the work listed at https://www.openphilanthropy.org/get-involved/jobs/analyst-specializing-potential-risks-advanced-artificial-intelligence) over time.

comment by Ben Pace · 2018-03-26T19:09:04.040Z · score: 1 (1 votes) · EA(p) · GW(p)

Thanks Holden!

comment by Risto_Uuk · 2018-03-26T17:28:47.224Z · score: 8 (8 votes) · EA(p) · GW(p)

If someone can't apply right now due to other commitments, do you expect there to be new roles for generalist research analysts next year as well? What are the best ways one could make oneself a better candidate meanwhile?

comment by HoldenKarnofsky · 2018-03-26T18:35:50.772Z · score: 4 (4 votes) · EA(p) · GW(p)

There will probably be similar roles in the future, though I can't guarantee that. To become a better candidate, one can accomplish objectively impressive things (especially if they're relevant to effective altruism); create public content that gives a sense for how they think (e.g., a blog); or get to know people in the effective altruism community to increase the odds that one gets a positive & meaningful referral.

comment by NatKozak · 2018-03-26T19:41:53.539Z · score: 7 (6 votes) · EA(p) · GW(p)

I'm graduating from the University of Michigan (Ann Arbor) with an Honors History degree this year. This gave me a nonstandard amount of experience with literature reviews and research for an undergraduate student. However, I've seen that OpenPhil generally prefers its candidates to have non-humanities majors.

1) Is the latter claim true?

2) In general, how does OpenPhil rate research experience that is not in the field(s) currently being explored?

comment by Jamie_Harris · 2018-04-04T15:25:57.920Z · score: 5 (4 votes) · EA(p) · GW(p)

Thanks for the question - I have wondered the same, as I also studied History at undergraduate level.

A slight detour from your question, but maybe of interest. There is currently is no community / FB group for people with backgrounds or research interests in History within EA that I know of. There have been quite a few times when discussions around the usefulness of historical studies has come up and it might be good to share ideas and collaborate.

I don't have time to try and coordinate this at the moment, but it seems like trying to establish some sort of discussion forum (or organisation?) for using the study of history to advance our understanding of (and strategy towards) cause areas which are often prioritised within EA.

If this is something you (or anyone else seeing this) has an interest in me developing, it's something to bear in mind? People should feel free to contact me at james_a_harris [at] hotmail.co.uk if you want to talk about it further.

Examples: I'm thinking primarily within Effective Animal Advocacy (Sentience Institute's study of the British antislavery movement; ACE discontinuing their social studies project; technology adoption being considered as a precedent for clean meat e.g. by Sentience Institute and Paul Shapiro) but this would also apply to other fields. The systematic approach described in the post linked at [1] seems to correlate more closely with the approach Holden and others took at OPP than it does the studies done in the Effective Animal Advocacy sphere.

[1] http://effective-altruism.com/ea/1lz/why_we_should_be_doing_more_systematic_research/

comment by MichaelPlant · 2018-04-06T23:39:35.088Z · score: 1 (1 votes) · EA(p) · GW(p)

just emailed you.

comment by Jamie_Harris · 2018-09-28T22:49:41.570Z · score: 1 (1 votes) · EA(p) · GW(p)

Michael, apologies for this. I just came back to check this post. I didn't ever receive the email because the formatting of the EA forum removed the underscore from my email, and I didn't notice at the time. If you can find the email that you sent from your sent box in April, and could forward it to me at jamie@sentienceinstitute.org that would be great!

comment by John_Maxwell (John_Maxwell_IV) · 2018-03-27T07:05:51.096Z · score: 3 (2 votes) · EA(p) · GW(p)

OpenPhil is big on history, I would apply.

comment by lukeprog · 2018-03-28T15:50:08.773Z · score: 1 (1 votes) · EA(p) · GW(p)

(I work for Open Phil.)

As Holden said elsewhere, "Quantitative aptitude is not a hard requirement for this position (there are some ways the role could evolve that would not require it), but it's a major plus." In that sense we prefer candidates with quantitative/technical majors, but also note that in most cases we pay more attention to an applicant's performance on the work tests that are part of our application process than we do to their CV.

Research experience in a current Open Phil focus area is a bonus, but less important than general quantitative aptitude and performance on our work tests.

comment by SiebeRozendal · 2018-03-26T17:31:24.453Z · score: 7 (7 votes) · EA(p) · GW(p)

A quick Google search gave me the impression that it isn't very easy to get a work visa in the US even though it will be sponsored. Is this correct, and do you have stronger requirements for non-US applicants because they'll be less likely to actually be able to work for you? (I'm completely unfamiliar with the visa system)

comment by HoldenKarnofsky · 2018-03-26T18:36:35.312Z · score: 2 (2 votes) · EA(p) · GW(p)

We don't control the visa process and can't ensure that people will get sponsorship. We don't expect sponsorship requirements to be a major factor for us in deciding which applicants to move forward with.

comment by rjmk · 2018-05-02T20:44:12.051Z · score: 2 (2 votes) · EA(p) · GW(p)

Thanks for the response. I understand OPP doesn't control the visa process, but do you have a rough sense of how likely a successful applicant would be to get a visa after being sponsored, or is it a complete unknown?

comment by lukeprog · 2018-05-02T22:43:33.362Z · score: 0 (0 votes) · EA(p) · GW(p)

Unfortunately the likelihood is still pretty unclear to us at this point, and the available options vary a fair bit by applicant, depending on which country they're from, whether they recently graduated from undergrad or graduate school, and other factors.

comment by lukeprog · 2018-04-19T03:04:58.375Z · score: 6 (5 votes) · EA(p) · GW(p)

Here's a question I received via email, which I'll answer here so others can benefit from the answer.

QUESTION

Would you be able to give examples of how the research has been used for decision making around giving money/grants?

ANSWER

Sure, here are a few examples:

  • Our research on the history of philanthropy helped us decide to be more ambitious: "we are more interested in working on daunting problems over long periods of time after learning about some of philanthropy’s past contributions." This has increased the amount of attention and funding we've spent on daunting challenges that will likely require investment over many years to achieve the impacts we hope for.
  • Our shallow investigations into several different global catastrophic risks enabled us to choose initial priorities in that category. Most importantly, we launched grantmaking focus areas in "biosecurity and pandemic preparedness" and "potential risks from advanced artificial intelligence." Across those two focus areas we've since made more than $80 million in grants.
  • Our research on moral patienthood persuaded us to begin making grants related to fish welfare. Since then, we've made more than $6 million in fish welfare grants.
comment by Xavier_ORourke · 2018-05-11T09:59:14.789Z · score: 5 (4 votes) · EA(p) · GW(p)

Thanks Holden and Luke for answering so many questions <3

Thinking in terms of broad generalisations/approximations - if you had to draw a graph depicting the value provided to OP by a new research analyst over time, what kind of shape would this graph have?

Or to ask the question in a different way: Are your efforts to hire for new OP roles motivated more by a desire to make better grants in the next couple of years, or by an intention to have a strong team in place several years from now which does high quality work in the future?

comment by lukeprog · 2018-05-11T21:11:10.151Z · score: 3 (3 votes) · EA(p) · GW(p)

Responding to your second formulation of the question, the answer is "more the latter than the former." We intend to invest heavily in training and mentoring new hires, and we hope that research analysts will end up being long-term core contributors at Open Phil — as research analysts, as grant investigators, and as high-level managers, among other roles — or, in some cases, in important roles that require similar skills outside Open Phil.

comment by jackva · 2018-04-01T14:44:05.861Z · score: 5 (5 votes) · EA(p) · GW(p)

Thanks, Holden, for the AMA and everyone for their questions!

When looking at the application form initially, there was a mentioning of an applied task as part of the application -- as far as I can tell this has now been removed.

Could someone confirm this change? Does that mean that the application now only consists of the Google Form plus resume in the first stage?

Thanks in advance!

comment by lukeprog · 2018-04-02T20:52:09.056Z · score: 4 (3 votes) · EA(p) · GW(p)

Yes, we recently removed a 2-question timed test from the job ad page. We wanted to lower the cost of submitting an initial application, and also we want to filter some applications before we request that applicants take the timed test. However, we are still using this timed test in an early part of our application process, and we still think it is diagnostic of who will be a good fit for our research analyst positions.

(I work for Open Phil.)

comment by jackva · 2018-04-03T13:30:09.173Z · score: 0 (0 votes) · EA(p) · GW(p)

Thanks, Luke!

comment by itty · 2018-03-26T18:18:01.912Z · score: 5 (4 votes) · EA(p) · GW(p)

Who is likely to manage new Grants Associates, Operations Associates, and Research Analysts?

comment by HoldenKarnofsky · 2018-03-26T18:41:35.104Z · score: 2 (2 votes) · EA(p) · GW(p)

All bios here: https://www.openphilanthropy.org/about/team

Grants Associates and Operations Associates are likely to report to Derek or Morgan. Research Analysts are likely to report to people who have been in similar roles for a while, such as Ajeya, Claire, Luke and Nick. None of this is set in stone though.

comment by SiebeRozendal · 2018-03-26T17:11:20.418Z · score: 5 (5 votes) · EA(p) · GW(p)

What are the working hours like for a position like Research Analyst? Strict/flexible? 40 hours/week or other? What is the overtime like on average, and what is it like on peaks?

comment by HoldenKarnofsky · 2018-03-26T17:49:45.765Z · score: 8 (7 votes) · EA(p) · GW(p)

We're flexible. People don't clock in or out; we evaluate performance based on how much people get done on a timescale of months. We encourage people to work hard but also prioritize work-life balance. The right balance varies by the individual.

Most people here work more than one would in a traditional 9-5 job. (A common figure is 35-40 "focused" hours per week.) I think that reflects that they're passionate about their work rather than that they feel pressure from management to work a lot. We regularly check in with people about work-life balance and encourage them to work less if it seems this would be good for their happiness.

comment by SiebeRozendal · 2018-03-26T16:50:33.953Z · score: 5 (4 votes) · EA(p) · GW(p)

It seems that OpenPhil wants a more satisfactory answer to moral uncertainty than just worldview diversification before ramping up the amount of grants per year. Is this part of why you are hiring new Research Analysts, and if so, how much will they work on this problem? (This seems like a very interesting but hard problem)

comment by HoldenKarnofsky · 2018-03-26T16:56:23.481Z · score: 5 (5 votes) · EA(p) · GW(p)

We could certainly imagine ramping up grantmaking without a much better answer. As an institution we're often happy to go with a "hacky" approach that is suboptimal, but captures most of the value available under multiple different assumptions.

If someone at Open Phil has an idea for how to make useful progress on this kind of question in a reasonable amount of time, we'll very likely find that worthwhile and go forward. But there are lots of other things for Research Analysts to work on even if we don't put much more time into researching or reflecting on moral uncertainty.

Also note that we may pursue an improved understanding via grantmaking rather than via researching the question ourselves.

comment by SiebeRozendal · 2018-03-26T17:16:06.942Z · score: 1 (1 votes) · EA(p) · GW(p)

I'm very curious about how that improved understanding would come about via grantmaking. Any write-up you have about this? I can see how you'd learn about tractability, and maybe about neglectedness, but I wonder how you incorporate this in your decision-making.

Anyway, this might go a little too off-topic so I'd understand if you replied to other questions first :)

comment by HoldenKarnofsky · 2018-03-26T19:38:52.687Z · score: 1 (1 votes) · EA(p) · GW(p)

I'm referring to the possibility of supporting academics (e.g. philosophers) to propose and explore different approaches to moral uncertainty and their merits and drawbacks. (E.g., different approaches to operationalizing the considerations listed at https://www.openphilanthropy.org/blog/update-cause-prioritization-open-philanthropy#Allocating_capital_to_buckets_and_causes , which may have different consequences for how much ought to be allocated to each bucket)

comment by HoldenKarnofsky · 2018-03-26T19:31:05.952Z · score: 4 (3 votes) · EA(p) · GW(p)

Keep in mind that Milan worked for GiveWell, not OP, and that he was giving his own impressions rather than speaking for either organization in that post.

That said:

*His "Flexible working schedule" point sounds pretty consistent with how things are here.

*We continue to encourage time tracking (but we don't require it and not everybody does it).

*We do try to explicitly encourage self-care.

Does that respond to what you had in mind?

comment by itty · 2018-03-26T18:33:00.722Z · score: 4 (4 votes) · EA(p) · GW(p)

The job description for Research Analyst says that the best candidates will have "comfort thinking in terms of expected value and using systematic, quantitative frameworks." How quantitative should a candidate be to apply? For example, if a person feels comfortable with basic expected value concepts but finds GiveWell's CEA overwhelming (and probably could never produce something similar to GiveWell's CEA), is that not quantitative enough?

comment by HoldenKarnofsky · 2018-03-26T19:15:26.525Z · score: 5 (4 votes) · EA(p) · GW(p)

GiveWell's CEA was produced by multiple people over multiple years - we wouldn't expect a single person to generate the whole thing :)

I do think you should probably be able to imagine yourself engaging in a discussion over some particular parameter or aspect of GiveWell's CEA, and trying to improve that parameter or aspect to better capture what we care about (good accomplished per dollar). Quantitative aptitude is not a hard requirement for this position (there are some ways the role could evolve that would not require it), but it's a major plus.

comment by casebash · 2018-03-28T13:59:58.275Z · score: 3 (3 votes) · EA(p) · GW(p)

What's Givewell's CEA?

comment by MarkusAnderljung · 2018-03-28T14:28:03.837Z · score: 5 (4 votes) · EA(p) · GW(p)

Cost-Effectiveness Analysis :)

comment by Mappy (Mario) · 2018-03-26T18:24:11.751Z · score: 4 (4 votes) · EA(p) · GW(p)

Is there any possibility for the research analyst (or some other) role to be taken up completely remotely? (i.e. while living in another country)

comment by lukeprog · 2018-03-26T18:49:50.751Z · score: 4 (4 votes) · EA(p) · GW(p)

Jumping in here for Holden (I'm an Open Phil RA)…

We would like our Research Analysts to work in the SF office most of the time, with 'partly remote' in the job ad meaning that they can occasionally work from home/different locations. Despite our policy, we aren't totally closed to the idea of hiring someone who mostly works remotely, but they'd have to meet a higher bar.

comment by MaxRa · 2018-04-19T10:34:30.659Z · score: 2 (1 votes) · EA(p) · GW(p)

Is it possible to switch to remote work (in my case in Europe) after working from office in SF e.g. for one year?

comment by lukeprog · 2018-04-21T02:46:22.505Z · score: 1 (1 votes) · EA(p) · GW(p)

Possibly. If a research analyst excels at the job after being hired, and seems likely to continue excel even while remote, we'd certainly consider it.

comment by gsastry · 2018-03-26T18:01:11.911Z · score: 3 (3 votes) · EA(p) · GW(p)

What key metrics do research analysts pay attention to in the course of their work? More broadly, how do employees know that they're doing a good job?

comment by HoldenKarnofsky · 2018-03-26T18:54:33.185Z · score: 1 (1 votes) · EA(p) · GW(p)

We do formal performance reviews twice per year, and we ask managers to use their regular (~weekly) checkins with reports to sync up on performance such that nothing in these reviews should be surprising. There's no unified metric for an employee's output here; we set priorities for the organization, set assignments that serve these priorities, set case-by-case timelines and goals for the assignments (in collaboration with the people who will be working on them), and compare output to the goals we had set.

comment by Khorton · 2018-03-26T17:22:17.549Z · score: 3 (3 votes) · EA(p) · GW(p)

I'm under the impression that holidays, sick days, and parental leave are on average much less generous in the US compared to Europe or Canada. What are your policies on time off work?

comment by HoldenKarnofsky · 2018-03-26T17:43:02.578Z · score: 3 (3 votes) · EA(p) · GW(p)

We're in the process of reviewing our policies, but we're likely to settle on something like 25 paid days off (including sick days), 10 holiday days (with the option to work on holidays and use the paid time off elsewhere), several months of paid parental leave, and a flexible unpaid leave policy for people who want to take more time off. We are also flexible with respect to working from home.

comment by Khorton · 2018-03-26T17:48:57.342Z · score: 0 (0 votes) · EA(p) · GW(p)

To clarify, are the 10 holiday days part of the 25 paid days off, or in addition to?

comment by Khorton · 2018-03-26T17:53:03.406Z · score: 0 (0 votes) · EA(p) · GW(p)

Or by 'holiday days' do you mean statutory holidays like Thanksgiving?

comment by HoldenKarnofsky · 2018-03-26T18:18:40.694Z · score: 1 (1 votes) · EA(p) · GW(p)

Yes, I mean statutory holidays like Thanksgiving.

comment by Denise_Melchin · 2018-03-27T17:25:39.964Z · score: 1 (1 votes) · EA(p) · GW(p)

It is still unclear to me whether the statutory holidays are supposed to be included in the 25 days paid days off or in addition to.

comment by Nick_Beckstead · 2018-03-27T18:49:58.863Z · score: 2 (2 votes) · EA(p) · GW(p)

In addition to, 35 days total. (I work at Open Phil.)

comment by Michael_Cannon · 2018-03-26T16:43:29.921Z · score: 3 (3 votes) · EA(p) · GW(p)

In my current field – biomedical research – there’s a tendency for the best researchers to be drawn towards important problem areas, but to then detour towards interesting problems instead of important ones when choosing a specific research topic. I think this is especially true for generalists, who often have a wider perspective on why the Really Cool problem is Really Cool. I’m guessing similar things happen in most fields. Does the Open Philanthropy Project try to combat this? As a Research Analyst, will the aim always be Important over Interesting?

Thanks! :)

comment by HoldenKarnofsky · 2018-03-26T16:50:16.634Z · score: 3 (5 votes) · EA(p) · GW(p)

We're certainly not using the same standards as academia! In general, we aim to base assignments on a combination of 1. How we judge what's most important to do (in terms of accomplishing as much good as possible) 2. What the employees themselves are motivated and interested to work on (including their own judgments of how to do as much good as possible).

comment by Michael_Cannon · 2018-03-26T17:38:17.296Z · score: 1 (1 votes) · EA(p) · GW(p)

Definitely not academia! Sounds like a healthy working environment, and I admire your focus on researcher autonomy.

comment by SoerenMind · 2018-03-26T21:13:22.756Z · score: 2 (2 votes) · EA(p) · GW(p)

How is doing research at Open Philanthropy different from doing research in academia? To make things concrete, how would the work of someone doing economic modelling or ML research in academia differ from typical research at OpenPhil?

comment by lukeprog · 2018-03-28T15:51:12.466Z · score: 6 (6 votes) · EA(p) · GW(p)

(I work for Open Phil.)

Comparing to academia, I'd say that research at Open Phil is (1) consistently targeted at what will help us do as much good as possible rather than what is most intellectually interesting or prestigious, (2) aimed at informing our actions as cheaply as possible, meaning that we cut corners when we don't think doing so will change our bottom-line conclusions, rather than trying to live up to the standards for thoroughness etc. expected in academia, and (3) only aimed at what academia would consider "novel research" when that's what is required to help us do the most good.

comment by MichaelPlant · 2018-03-29T10:45:02.359Z · score: 5 (5 votes) · EA(p) · GW(p)

Wow. Working at Open Phil sounds like a dream compared to academia. You've identified three things I spend huge amounts of time doing as part of my research and find intensely irritatingly.

comment by tessa · 2018-03-26T18:17:21.607Z · score: 2 (2 votes) · EA(p) · GW(p)

How much collaboration exists between research analysts (or operations associates, for that matter)?

I decided against working in academic research because I do much better in a team environment (short feedback loops, bouncing ideas off peers, sense that my work contributes to shared purpose and projects) than I do working independently. I prefer the industry side of basically all of Philip Guo's industry vs. academia comparisons. Would it still make sense for me to apply for an OpenPhil job? I think I have relevant skills, but I'm worried that I wouldn't be effective in a research environment, even if it is non-academic.

comment by HoldenKarnofsky · 2018-03-26T18:34:06.552Z · score: 3 (3 votes) · EA(p) · GW(p)

Most of the roles here involve a lot of independent work, consisting of reading/writing/analysis and one-on-one checkins rather than large-group collaboration. It’s possible that this will change for some roles in the future (e.g. it’s possible that we’ll want more large-group collaboration as our cause prioritization team grows), but we’re not sure of that. I think you should probably be prepared for a fair amount of work along the lines of what I've described here.

comment by E_Ilin · 2018-03-26T18:12:06.046Z · score: 2 (2 votes) · EA(p) · GW(p)

Hi Holden,

the job description for Research Analysts says:

Research Analysts will receive intensive training and mentorship, and over time will become highly experienced with our approach to reasoning transparency, cost-effectiveness analysis, critical evidence assessment, grant investigation, and balancing thoroughness with efficiency."

Could you please elaborate a bit more on the training and mentorship part? E.g.: How much time is reserved for training? Who would be the mentors? What would the relation with the mentor(s) be like?

Thanks for offering this Q&A!

comment by HoldenKarnofsky · 2018-03-26T18:28:05.739Z · score: 3 (3 votes) · EA(p) · GW(p)

I answered a similar question here: http://effective-altruism.com/ea/1mf/hi_im_holden_karnofsky_ama_about_jobs_at_open/dpl

In general, people who have been in the Research Analyst role for a while will be the managers and primary mentors of new Research Analysts. There will be regular (~weekly) scheduled checkins as well as informal interaction as needed (e.g., over Slack).

There's no hard line between training and "just doing the work" - every assignment should have some direct value and some training value. We expect to lean pretty hard toward the training end of the spectrum for people's first few months, then gradually move along the spectrum to where assignments are more optimized for direct value.

comment by JanBrauner · 2018-03-26T17:07:37.425Z · score: 2 (2 votes) · EA(p) · GW(p)

Hey Holden, thanks for doing this. Suppose I applied for the research analyst position and didn't get it. Which of the following would then be more likely to eventually land me a job at OPP, and how much more likely (assuming I would perform well in both)?

a) becoming research analyst at GiveWell

b) doing research in one of OPP's focus areas (biosecurity/AI safety).

comment by HoldenKarnofsky · 2018-03-26T17:11:18.662Z · score: 2 (2 votes) · EA(p) · GW(p)

I would rate those about equally, though I'd add that GiveWell would prefer not to hire people whose main goal is to go to OP.

comment by branperr · 2018-03-26T16:41:45.170Z · score: 2 (2 votes) · EA(p) · GW(p)

What's daily work and office culture look like in the OPP? Do you all host group activities, happy hours, or anything like that outside of working hours?

comment by HoldenKarnofsky · 2018-03-26T16:59:20.032Z · score: 4 (4 votes) · EA(p) · GW(p)

We currently have a happy hour every 3 weeks and host group activities as well, including occasional parties and a multiple-day staff retreat this year. We want to make it easy for staff to socialize and be friends, without making it a requirement or an overly hard nudge (if people would rather stick to their work, that's fine by us).

comment by lukeprog · 2018-05-23T21:01:43.745Z · score: 1 (1 votes) · EA(p) · GW(p)

Somebody asked the following question:

I understand that OP’s preference is to have promising candidates attend an in-person work trial rather than do an additional remote work test; would this preference still stand if the candidate in question has to obtain a US work visa sponsorship in order to attend the in-person trial?

Our reply is: Yes, that preference stands regardless of current work authorization status, though of course in some cases there won't be any way for us to help an applicant get US work authorization, depending on their situation.

comment by Ahmad · 2018-05-25T17:50:39.930Z · score: 0 (0 votes) · EA(p) · GW(p)

Thanks for the answer, Luke. I removed the question from here after I reread the "Update on Open Philanthropy research analyst recruiting for 2018" file sent earlier and found the answer there!

comment by markm · 2018-05-02T13:34:56.290Z · score: 1 (1 votes) · EA(p) · GW(p)

1a. Has Open Phil set any aggressive org-wide goals or timelines for 2018?
1b. The published plan for 2018 says that OPP expects to give "well over $100 million" [1]. What is this expectation based on? Or is it a goal?

2a. Other than current funders, who considers the research coming from Open Phil RAs to be reputable, credible and useful? (ie. government?)
2b. Does it matter that RAs aren't PhDs or that Open Phil isn't directly affiliated with any educational institution?

[1] https://www.openphilanthropy.org/blog/our-progress-2017-and-plans-2018

comment by lukeprog · 2018-05-02T22:43:15.441Z · score: 1 (1 votes) · EA(p) · GW(p)

Quick replies to each:

1a. Our goals for 2018 are laid out in the post you linked to.

1b. The expectation is based mostly on the fact that we gave well over $100 million last year, and we're devoting similar time and effort to grantmaking in 2018.

2a. Open Phil is still a fairly new organization, and I don't think many know much about us yet. Probably we are best known in the effective altruism community, where we seem to have a strong reputation.

2b. Does it matter for our reputation, do you mean? I'm not sure. I'm not aware of us having received critiques about that.

comment by markm · 2018-05-03T13:41:23.844Z · score: 0 (0 votes) · EA(p) · GW(p)

The purpose of these questions was to better estimate if an RAs impact can be expected to increase, decrease, or remain the same in the coming years.

An aggressive measurable goal (ie. increase estimated QALYs gained by a factor of x) would indicate to me that an RAs expected impact would increase. (It's possible that a measurable goal might be trivial to set because the error bars might be too large. I don't know enough to know.)

If other funders (esp. big funders such as government) considered Open Phil research credible enough to base their decisions on, that would also indicate more expected impact. ie. already published research would be reused in the future by other large donors to effectively allocate more funds.

Either way, it seems that an RAs expected impact is higher than many other career alternatives, even if it decreases a bit in the next few years.

comment by Khorton · 2018-04-12T23:29:36.095Z · score: 1 (1 votes) · EA(p) · GW(p)

What are the stages of the application process? What proportion of applicants do you expect to filter out at each stage?

comment by lukeprog · 2018-04-13T21:44:31.444Z · score: 3 (3 votes) · EA(p) · GW(p)

Because applicants are working through our application process at different speeds, we're still learning what portion of applicants are invited forward at each stage. We are also adjusting the process as we go along. As of today, our application process looks like this:

  1. Initial submission of application + résumé.
  2. 2-question timed test.
  3. A "conversation notes" work test. (compensated via honorarium)
  4. A brief call, to explain the rest of our process and answer the applicant's questions.
  5. An "internal grant write-up" work test. (compensated via honorarium)

To some degree we are still determining next steps after #5, and they depend somewhat on the applicant's availability and preferences, and that's the main thing we discuss with applicants at step #4, on an individual basis.

comment by alexflint · 2018-03-27T18:27:55.247Z · score: 1 (1 votes) · EA(p) · GW(p)

What advice would you give to a potential OpenPhil hire re how to think about the value of joining OpenPhil full time vs starting an organization that OpenPhil would be likely to fund vs joining an organization that has already received funding from OpenPhil?

comment by lukeprog · 2018-03-30T15:19:49.977Z · score: 0 (0 votes) · EA(p) · GW(p)

(I work for Open Phil.)

This is tough to answer in general; see my reply to a similar question here. Probably best to discuss this on an individual basis if/when an applicant reaches the interview stage of our application process.

comment by SiebeRozendal · 2018-03-27T18:10:56.873Z · score: 1 (1 votes) · EA(p) · GW(p)

Is there a standard contract length you will offer RA's if there is no trial period?

comment by lukeprog · 2018-03-30T15:19:29.686Z · score: 0 (0 votes) · EA(p) · GW(p)

(I work for Open Phil.)

For salaried positions there is no "contract length"; employment at Open Philanthropy is at-will, just as it is with most California employers.

comment by scarves · 2018-03-27T00:26:02.612Z · score: 1 (1 votes) · EA(p) · GW(p)

I'm curious whether Open Phil has done any estimates of the relative impact of someone who goes to work for you vs someone who does direct work in one of your focus areas? All else equal (i.e., assume personal fit is the same, etc.), which option might be better?

comment by lukeprog · 2018-03-28T15:53:34.600Z · score: 0 (0 votes) · EA(p) · GW(p)

(I work for Open Phil.)

This is tough to answer in general. It would depend hugely on the focus area, their role in that focus area, their fit for Open Phil research analyst work, etc. I can easily think of people who I think are doing more good by working in a focus area, and I can easily think of other people who would be valuable in a focus area but even more valuable as a generalist research analyst at Open Phil.

comment by Milan_Griffes · 2018-03-26T20:51:33.120Z · score: 1 (1 votes) · EA(p) · GW(p)

What are the most important ways in which Open Phil's culture is different from GiveWell's culture?

comment by lukeprog · 2018-03-28T15:53:06.036Z · score: 0 (0 votes) · EA(p) · GW(p)

The core values and norms are very similar, so specific differences are relatively hard to articulate/pinpoint (which isn't to say they aren't there).

comment by Milan_Griffes · 2018-03-28T18:32:02.062Z · score: 0 (0 votes) · EA(p) · GW(p)

Are you saying that Open Phil & GiveWell cultures are different in hard-to-articulate but important ways? Or that the cultures are basically the same in any way that matters?

comment by lukeprog · 2018-03-30T15:20:16.265Z · score: 1 (1 votes) · EA(p) · GW(p)

It's hard to say. To me the cultures seem very similar, but my personal access to GiveWell culture is limited since I work for Open Phil. Perhaps to other people the cultures seems importantly different, but I can't predict what those perceived differences would be.

comment by Ikaxas · 2018-03-26T19:55:07.333Z · score: 1 (1 votes) · EA(p) · GW(p)

Hi,

What would be the attitude towards someone who wanted to work with you after undergrad for a year or two, but then go on to graduate school (likely for philosophy in my case), with an eye towards then continuing to work with you or other EA orgs after grad school?

comment by lukeprog · 2018-03-28T15:53:58.088Z · score: 0 (0 votes) · EA(p) · GW(p)

(I work for Open Phil.)

We'd encourage someone like that to apply, and flag in their application that they currently would plan to leave after a couple years to go to graduate school. Depending on their fit for the work, we might be excited to consider their application anyway, or we might not.

comment by NatKozak · 2018-03-26T19:44:12.740Z · score: 1 (1 votes) · EA(p) · GW(p)

Do you recommend any textbooks (or other resources) that might help an applicant become more familiar with the expected value frameworks used at OpenPhil?

comment by lukeprog · 2018-03-28T15:50:42.339Z · score: 1 (1 votes) · EA(p) · GW(p)

(I work for Open Phil.)

Spending time with GiveWell's CEA is probably more helpful than reading a textbook. You could also just practice doing Fermi estimates in general.

comment by SiebeRozendal · 2018-03-26T16:42:45.312Z · score: 1 (1 votes) · EA(p) · GW(p)

Hi Holden, nice initiative.

I have a question about the Research Analyst role. How generalist will they be? I can imagine them being somewhat focused on one or two focus areas besides more general issues such as how to implement moral uncertainty practically.

comment by HoldenKarnofsky · 2018-03-26T16:51:25.990Z · score: 1 (1 votes) · EA(p) · GW(p)

This varies by the individual. We have some Research Analysts who are always working on a variety of things, and some who have become quite specialized. It varies largely by the interests/preferences of the employee.

comment by Khorton · 2018-03-26T16:41:19.797Z · score: 1 (1 votes) · EA(p) · GW(p)

I really appreciate that your application form is so short, but what's the logic behind focusing on applicant's undergraduate degrees? What are you assessing in the application?

comment by HoldenKarnofsky · 2018-03-26T16:52:37.039Z · score: 3 (3 votes) · EA(p) · GW(p)

All else equal, we consider applicants stronger when they have degrees in challenging fields from strong institutions. It’s not the only thing we’re looking at, even at that early stage. And the early stage is for filtering; ultimately, things like work trial assignments will be far more important to hiring decisions.

comment by Khorton · 2018-03-26T19:49:37.218Z · score: 2 (2 votes) · EA(p) · GW(p)

What does a "challenging field" mean here? Do you have a ranking of fields by degree of challenge or rely mainly on intuition?

comment by SiebeRozendal · 2018-03-26T17:02:43.376Z · score: 1 (1 votes) · EA(p) · GW(p)

Not sure if interpreting Khorton correctly, but interested anyway: Why focus on undergrad and not on postgrad (or highest level achieved/pursuing)?

comment by HoldenKarnofsky · 2018-03-26T17:16:11.082Z · score: 6 (6 votes) · EA(p) · GW(p)

Hm, I'm not sure why our form asks for more detail on undergrad relative to grad - we copied the form from GiveWell and may not have thought about it. It's possible this is because the form was being used in an earlier GiveWell search where few applicants had been to grad schools. I'll ask around about this.

comment by branperr · 2018-03-26T16:39:30.363Z · score: 1 (1 votes) · EA(p) · GW(p)

How do you enjoy living in/near and working San Francisco? How's the commute, the expensive housing, and all that affected your lives?

comment by Nick_Beckstead · 2018-03-26T18:33:12.720Z · score: 2 (2 votes) · EA(p) · GW(p)

I am a Program Officer at Open Philanthropy who joined as a Research Analyst about 3 years ago.

The prior two places I lived were New Brunswick, NJ and Oxford, UK. I live in a house with a few friends. It is 25-30m commute door-to-door via BART. My rent and monthly expenses are comparable to what I had in Oxford but noticeably larger than what I had in New Brunswick. I got pay increases when I moved to Open Phil, and additional raises over time. I’m comfortable on my current salary and could afford to get a single-bedroom apartment if I wanted, but I’m happy where I am.

Overall, I would say that it was an easy adjustment.

comment by Benjamin_Todd · 2018-03-27T04:35:31.510Z · score: 2 (2 votes) · EA(p) · GW(p)

Surely rent is much higher than Oxford on average? It's possible to get a great place in Oxford for under £700 per month, while comparable in SF would be $1300+. Food also seems about 30% more expensive, and in Oxford you don't have to pay for a commute. My overall guess is that $80k p.a. in SF is equivalent to about £40k p.a. in Oxford.

comment by Nick_Beckstead · 2018-03-27T17:47:57.691Z · score: 0 (0 votes) · EA(p) · GW(p)

I don't mean to make a claim re: averages, just relaying personal experience.

comment by AGB · 2018-03-27T17:39:24.630Z · score: 0 (0 votes) · EA(p) · GW(p)

To chime in as someone who has very recently spent a lot of time in both London and SF, a 1.8:1 ratio (as in $1.8y is about the same as £y) is very roughly what I would have said for living costs between that pair, though living circumstances will vary significantly.

Pound to dollar exchange rates have moved a ton in the last few years, whereas I don't think local salaries or costs of living have moved nearly as much, so I expect that 1.8:1 heuristic to be more stable/useful than trying to do the same comparison including a currency conversion (depending what point in the last few years you picked/moved, that ratio would imply anywhere between a 1.05x increase and a 1.55x increase).

comment by HoldenKarnofsky · 2018-03-26T17:19:52.808Z · score: 2 (2 votes) · EA(p) · GW(p)

Perhaps other staff will chime in here, but my take: our pay is competitive and takes cost of living into account, and we are near public transportation, so I don't think the rents or commutes are a major issue. As a former NYC resident, I think the Bay Area is a great place to live (weather, food, etc.) and has a very strong effective altruist community. I don't see a lot of drawbacks to living here if you can make it work.

comment by rileyharris · 2018-04-25T05:28:05.862Z · score: 0 (0 votes) · EA(p) · GW(p)
  • If someone was looking to work for OPP would an honours* or masters program be more beneficial than an undergraduate degree?

  • Are there particular questions or areas that could be worked on for a research project in honours/masters that are particularly helpful directly or develop the right kinds of skills for OPP? (especially in economics, philosophy or cognitive science)

  • ("Honours" in Australia is a 1 year research/coursework program)

comment by lukeprog · 2018-04-26T05:08:17.979Z · score: 0 (0 votes) · EA(p) · GW(p)

Completion of an honours or masters program provides us with a bit more evidence about an applicant's capabilities than an undergraduate degree does, but both are less informative to us than the applicant's performance on the various work samples that are part of our application process.

Because our roles are so "generalist," there are few domains that are especially relevant, though microeconomics and statistics are two unusually broadly relevant fields. In general, we find that those with a STEM background do especially well at the kind of work we do, but a STEM background is not required. A couple other things that are likely helpful for getting and excelling in an Open Phil research analyst role are calibration training and practice making Fermi estimates [LW · GW].

comment by Calvin_Baker · 2018-04-15T01:00:23.189Z · score: 0 (0 votes) · EA(p) · GW(p)

Is there any room in the application process for applicants to submit samples of original research or academic letters of recommendation?

Thank you!

comment by lukeprog · 2018-04-15T01:13:08.633Z · score: 0 (0 votes) · EA(p) · GW(p)

Yes, you may submit a writing sample by sending it to jobs@openphilanthropy.org, as FirstName.LastName.Sample (e.g. John.Smith.Sample.doc or John.Smith.Sample.pdf). If you'd like to submit a letter of recommendation, please include it as a page of your résumé.

Please keep in mind that writing samples and letters of recommendation are entirely optional, so if you don't already have them handy, I don't recommend spending time pulling them together. Our application process puts much more weight on work test performance anyway.

comment by KevinWatkinson · 2018-04-13T18:01:32.507Z · score: 0 (0 votes) · EA(p) · GW(p)

How does Open Philanthropy weigh conformity against talent?

comment by lukeprog · 2018-04-13T21:45:44.156Z · score: 0 (0 votes) · EA(p) · GW(p)

What kind of conformity are you asking about? Certainly, some degree of alignment with our mission and values is important to us, and so is talent and "fit" for the work. Our team members are encouraged to focus on optimizing for Open Phil's mission, even when it means pushing back on their manager.

comment by KevinWatkinson · 2018-04-14T08:45:41.610Z · score: 0 (0 votes) · EA(p) · GW(p)

Thanks for the response, yes I was wondering about conformity in the sense of prevailing thinking within a particular cause area. Is there an expectation for talent to conform to prevailing thinking to a certain degree and would this then reinforce that idea of being talented, or could talent be more related to a set of core values or principles?

I think some cause areas seem to have fairly high expectations of conformity toward in-group / out-group identity, so if this is the case then talented people may conform or not (given the assumption that not all talented people would necessarily be in-group thinkers), but it seems to confer various advantages on those that do.

comment by itty · 2018-03-26T18:14:34.021Z · score: 0 (0 votes) · EA(p) · GW(p)

GiveWell is also hiring for several roles right now. How should a person decide whether to apply to GiveWell or Open Phil? Are there significant differences in the work culture? Do Research Analysts at each org take on similar types of tasks?

comment by HoldenKarnofsky · 2018-03-26T18:29:56.686Z · score: 1 (1 votes) · EA(p) · GW(p)

They're different organizations and I don't know nearly as much about the GiveWell role. One big difference is the causes we work on.

If you're interested in both, I'd recommend applying to both, and if you are offered both roles, there will be lots of opportunities to learn more about each at that point in order to inform the decision.

comment by branperr · 2018-03-26T16:37:07.733Z · score: 0 (0 votes) · EA(p) · GW(p)

Hi there! Thank you for doing this. Is there a place for people who have just or recently finished their undergrad to help in your AI governance and policy position?

comment by HoldenKarnofsky · 2018-03-26T16:40:56.391Z · score: 2 (2 votes) · EA(p) · GW(p)

I'd recommend that recent grads looking to help with AI governance and policy apply for the Research Analyst position. With Research Analysts, we'll first focus on mentorship & training, then try to figure out where everyone can do the most good based on their interests and skills. Someone who has a high aptitude & interest for AI strategy would likely end up putting substantial time into that within a year or so (maybe less).

You can also check out roles at the Future of Humanity Institute.

comment by branperr · 2018-03-26T16:46:23.545Z · score: 2 (2 votes) · EA(p) · GW(p)

Thank you for your reply Holden! I submitted my application yesterday after hearing you all were talent constrained, so I hope to hear from you all sometime soon. What does the training process look like for people who are hired on as research analysts?

comment by HoldenKarnofsky · 2018-03-26T17:15:46.173Z · score: 1 (1 votes) · EA(p) · GW(p)

Broadly speaking, we're going to try to give people assignments that are relevant to our work and that we think include a lot of the core needed skills - things like evaluating a potential grant (or renewal) and writing up the case for or against. We'll evaluate these assignments, give substantial feedback, and iterate so that people improve. We'll also be providing resources for gaining background knowledge, such as "flex time," recommended reading lists and optional Q&A sessions. We've seen people improve a lot in the past and become core contributors, and think this basic approach is likely to lead to more of that.

comment by casebash · 2018-03-28T13:56:39.197Z · score: -2 (2 votes) · EA(p) · GW(p)

Meta: It might be good to announce AMA's in advance so that more people know to be online at that time.

comment by lukeprog · 2018-03-28T15:54:52.573Z · score: 2 (2 votes) · EA(p) · GW(p)

This one was announced here.