Launching the EAF Fund

post by stefan.torges (storges) · 2018-11-28T17:13:42.285Z · score: 59 (32 votes) · EA · GW · 12 comments

Contents

  Summary
  Mission
  Priority areas
    Tier 1
    Tier 2
    Past grants
  Fund management
    Fund managers
    Grantmaking
  When should you give to this fund?
  How to donate to this fund
None
12 comments

The Effective Altruism Foundation is launching the EAF Fund (a.k.a CLR Fund), a new fund focused on reducing s-risks. In this post we want to outline its mission, likely priority areas, and fund management structure. We also explain when it makes sense to donate to this fund.

Summary

Mission

The fund’s focus is on improving the quality of the long-term future by supporting efforts to reduce the worst s-risks from advanced artificial intelligence. (edited for clarity; see comment section)

Priority areas

Based on this mission, we have identified the following priority areas, which may shift as we learn more.

Tier 1

Tier 2

Past grants

Fund management

As we learn more, we might make changes to this initial setup.

Fund managers

We chose the fund managers based on their familiarity with the fund’s mission and prioritization, the amount of time they can dedicate to this work, and relevant research expertise. They were approved by the board of EAF.

Grantmaking

The current balance of the fund is $68,638 (as of November 27), and we expect to be able to allocate $400k–$1.5M during the first year. We will likely try different mechanisms for proactively enabling the kind of research we’d like to see, e.g. requests for proposals, prizes, teaching buy-outs, and scholarships.

Given the current state of academic research on s-risks, it’s impossible to find senior academic scholars who could judge the merit of a proposal based on its expected impact. However, we will consult domain experts where we think their judgment adds value to the evaluation. We also ran a hiring round [EA · GW] for a research analyst, whom we expect to support the fund managers. They may also take on more grantmaking responsibilities over time.

Grant recipients may be charitable organizations, academic institutions, or individuals. However, we expect to often fund individual researchers and small groups as opposed to large organizations or institutes. Grants are approved by a simple majority of the fund managers. We expect grants to be made at least every six months.

We will experiment with different formats for publishing our reasoning behind individual grant decisions and evaluating past grants (e.g. trying to use predictions). This will likely depend on the number and size of grants.

When should you give to this fund?

CEA has already written up reasons for giving to funds in general. We won’t repeat them here. So when does it make sense to give to this fund in particular?

When does it make sense to give to this fund instead of the EA Long-term Future Fund?

How to donate to this fund

You can donate to this fund via the Effective Altruism Foundation (donors from Germany, Switzerland, the Netherlands) or the EA Funds Platform (donors from the US or the UK).

Note: Until December 29 donations to the EAF Fund can be matched 1:1 as part of a matching challenge. (For the matching challenge we’re still using the former name “REG Fund”.)

12 comments

Comments sorted by top scores.

comment by Jan_Kulveit · 2018-11-28T22:32:23.892Z · score: 23 (14 votes) · EA(p) · GW(p)

Just wanted to note that the use of "worst case" in the mission statement

The fund’s mission is to address worst-case risks (s-risks) from artificial intelligence.

is highly non-intuitive for people with different axiology. Quoting from the s-risk explanation

For instance, an event leading to a future containing 10^35 happy individuals and 10^25 unhappy ones, would constitute an s-risk

At least for me, this would be a pretty amazing outcome, and not something which should be prevented.

In this context

We aim to differentially support alignment approaches where the risks are lowest. Work that ensures comparatively benign outcomes in the case of failure is particularly valuable from our perspective

sounds worrisome: do I interpret it correctly that in the ethical system held by the fund human extinction is comparatively benign outcome in comparison with risks like creation of 10^25 unhappy minds even if they are offset by much larger number of happy minds?

comment by Lukas_Gloor · 2018-11-28T23:13:14.529Z · score: 22 (12 votes) · EA(p) · GW(p)
At least for me, this would be a pretty amazing outcome, and not something which should be prevented.

Yeah, we're going to change the part that equates "worst case" with "s-risks". Your view is common and reflects many ethical perspectives.

We were already thinking about changing the definition of "s-risk" based on similar feedback, to make it more intuitive and cooperative in the way you describe. It probably makes more sense to have it refer to only the few % of scenarios where most of the future's expected suffering comes from (assuming s-risks are indeed heavy-tailed). These actual worst cases are what we want to focus on with the fund.

do I interpret it correctly that in the ethical system held by the fund human extinction is comparatively benign outcome in comparison with risks like creation of 10^25 unhappy minds even if they are offset by much larger [say 10^10x larger] number of happy minds?

No, that's incorrect. Insofar as some fund managers hold this view personally (e.g., I do, while Jonas would agree with you that the latter outcome is vastly better), it won't affect decisions because in any case, we want to avoid doing things that are weakly positive on some plausible moral views and very negative on others. But I can see why you were concerned, and thanks for raising this issue!

comment by Ozzie Gooen (oagr) · 2018-11-28T23:52:02.276Z · score: 6 (6 votes) · EA(p) · GW(p)

I guess to me, the part of the future with 10^25 unhappy individuals sounds like an s-risk. I would imagine an s-outcome could take place in a universe that's still on the net good. Just because the universe may be on the net good though, doesn't mean we shouldn't be concerned with large s-outcomes that may happen.

comment by Lukas_Gloor · 2018-11-29T00:18:47.355Z · score: 8 (6 votes) · EA(p) · GW(p)

Yeah. I put it the following way in another post [EA · GW]:

Especially when it comes to the prevention of s-risks affecting futures that otherwise contain a lot of happiness, it matters a great deal how the risk in question is being prevented. For instance, if we envision a future that is utopian in many respects except for a small portion of the population suffering because of problem x, it is in the interest of virtually all value systems to solve problem x in highly targeted ways that move probability mass towards even better futures. By contrast, only few value systems (ones that are strongly or exclusively about reducing suffering/bad things) would consider it overall good if problem x was “solved” in a way that not only prevented the suffering due to problem x, but also prevented all the happiness from the future scenario this suffering was embedded in.

So it'd be totally fine to address all sources of unnecessary suffering (and even "small" s-risks embedded in an otherwise positive future) if there are targeted ways to bring about uncontroversial improvements. :) In practice, it's sometimes hard to find interventions that are targeted enough because affecting the future is very very difficult and we only have crude levers. Having said that, I think many things that we're going to support with the fund are actually quite positive for positive-future-oriented value systems as well. So there certainly are some more targeted levers.

There are instances where it does feel justified to me to also move some probability mass away from s-risks towards extinction (or paperclip scenarios), but that should be reserved either for uncontroversially terrible futures, or for those futures where most of the disvalue for downside-focused value systems comes from. I doubt that this includes futures where 10^10x more people are happy than unhappy.

And of course positive-future-oriented EAs face analogous tradeoffs of cooperation with other value systems.

comment by Ozzie Gooen (oagr) · 2018-11-28T22:00:45.665Z · score: 17 (8 votes) · EA(p) · GW(p)

Happy to see this be set up! It makes a lot of sense to me that our community gradually gets more and more competition from different funds. Over time we'll get evidence on which one seem to perform the best.

A few questions:

1) Are you planning on coordinating information and decisions with other funds? I general my impression is that there should be similar application procedures and perhaps some information sharing. That said, I could imagine ways where coordination could come across in bad ways.

2) Where do you expect to get most of your donations from? Individual EA donors?

3) Your Tier-1 and Tier-2 questions don't seem exclusive to s-risks, they seem to me like things that would be very applicable to much of AI-risk concerns. Does this sound correct to you? Are you seeking projects that focus specifically on s-risks within these areas?

comment by Jonas Vollmer · 2018-11-28T23:03:34.488Z · score: 7 (7 votes) · EA(p) · GW(p)

Thanks! :)

1) The Long-Term Future Fund seems most important to coordinate with. Since I'm both a fund manager at the EAF Fund and an advisor to the Long-Term Future Fund, I hope to facilitate such coordination.

2) Individual EA donors, poker pros (through our current matching challenge), and maybe other large donors.

3) Yes, that sounds correct. We're particularly excited to support researchers who work on specific s-risk-related questions within those areas, but I expect that the research we fund could also positively influence AI in other ways (e.g. much of the decision theory work might make positive-sum trade more likely and could thereby increase the chance of realizing the best possible outcomes). We might also fund established organizations like MIRI if they have room for more funding.

comment by SiebeRozendal · 2018-12-04T07:54:53.999Z · score: 3 (3 votes) · EA(p) · GW(p)

Great to see this set up!

Small note on the name: is not at all clear for people unfamiliar with EAF what the fund is about, and a name change for the fund would probably get you more attention. Something like "suffering prevention fund", "suffering reduction fund", "long-term suffering reduction fund", "suffering risk fund" would all be significantly clearer (even though they feel inadequate to describe the fund's goals).

comment by Jonas Vollmer · 2018-12-04T09:44:11.321Z · score: 2 (2 votes) · EA(p) · GW(p)

Thanks for the feedback! I think part of the challenge is that the name also needs to be fairly short and easy to remember. "Long-Term Future Fund" is already a bit long and hard to remember (people often seem to get it wrong), so I'm nervous about making it even longer. We seriously considered "S-risk Fund" but ultimately decided against because it seems harder to fundraise for from people who are less familiar with advanced EA concepts (e.g., poker pros interested in improving the long-term future). Also, most people who understand the idea of s-risks will also know that EAF works on them.

I'd be curious to hear whether the above points were convincing, or whether you'd still perceive it as suboptimal.

comment by SiebeRozendal · 2018-12-17T13:37:49.780Z · score: 2 (2 votes) · EA(p) · GW(p)

I still perceive it as suboptimal, although I understand you don't like any of the potential names.

We seriously considered "S-risk Fund" but ultimately decided against because it seems harder to fundraise for from people who are less familiar with advanced EA concepts (e.g., poker pros interested in improving the long-term future).

I think this touches on a serious worry with fundraising from people unfamiliar with EA concepts: why should they donate to your fund rather than another EA fund if they don't understand the basic goal you are aiming for with your fund?

I can imagine that people donate to the EAF fund for social reasons (e.g. you happen to be well-connected to poker players) more than intellectual reasons (i.e. funders donate because they prioritize s-risk reduction). If that is the case, I'd find it problematic that the fund is not clearly named: it makes it less likely that people donate to particular funds for the right reasons.

Of course, this is part of a larger coordination problem in which all kinds of non-intellectual reasons are driving donation decisions. I am not sure what the ideal solution is*, but I wanted to flag this issue.

*Perhaps it should be a best practice for EA fundraisers to always recommend funders to all go through a (to be created) donation decision tool that takes them through some of the relevant questions. That tool would be a bit like the flowchart from the Global Priorities Project, but more user-friendly.

comment by Jonas Vollmer · 2018-12-17T14:17:40.229Z · score: 5 (4 votes) · EA(p) · GW(p)
why should they donate to your fund rather than another EA fund if they don't understand the basic goal you are aiming for with your fund?

We always point out that the fund is focused on reducing suffering in the long-term future.

Also, why should they donate to that other fund instead? E.g., the Long-Term Future Fund is also importantly motivated by "astronomical waste" type considerations which those donors don't understand either, and might not agree with.

I can imagine that people donate to the EAF fund for social reasons (e.g. you happen to be well-connected to poker players) more than intellectual reasons (i.e. funders donate because they prioritize s-risk reduction).

Yeah. This will always be the case with many donors, regardless of which fund they donate to.

Of course, this is part of a larger coordination problem in which all kinds of non-intellectual reasons are driving donation decisions.

I wouldn't call it a coordination problem in the game-theoretic sense, and I think in many cases this actually isn't even a problem: I think it's important that donors aren't deceived into supporting something that they wouldn't want to support; but in the many cases where donors don't have informed opinions (e.g., on population ethics), it's fine if you fill in the details for them with a plausible view held by a significant part of the community.

Perhaps it should be a best practice for EA fundraisers to always recommend funders to all go through a (to be created) donation decision tool that takes them through some of the relevant questions.

I think we'd be open to doing something like this.

comment by SiebeRozendal · 2018-12-18T13:45:23.002Z · score: 2 (2 votes) · EA(p) · GW(p)
We always point out that the fund is focused on reducing suffering in the long-term future.
Also, why should they donate to that other fund instead? E.g., the Long-Term Future Fund is also importantly motivated by "astronomical waste" type considerations which those donors don't understand either, and might not agree with.

Yes, I'm not saying you're misleading your donors, nor that they are less informed than donors of other funds. Just that there are many reasons people are donating to a particular fund, and I think properly naming a fund is a step in the right direction.

I wouldn't call it a coordination problem in the game-theoretic sense

I see it as coordination between different fund managers, where each wants to maximize the amount of funds for their own fund. As such, there are some incentives to not maximally inform one's donors if other funding possibilities, or the best arguments against donating to the fund they are fundraising for.

I'm not saying that this type of selfish behavior is very present in the EA community - I've heard that it is quite the opposite. But I do think that the situation is not yet optimal: current allocation of resources is not based largely on careful weighing the relevant evidence and arguments. I also think we can move closer to this optimal allocation.

Anyway, I didn't mean to make this into a large debate :) I'm glad the fund exists and I'd be happy if the name changes to something more distinguishable!

comment by Jonas Vollmer · 2018-12-20T21:28:52.893Z · score: 1 (1 votes) · EA(p) · GW(p)

Thanks! I see the point better now. While I don't fully agree with everything, I think it could make sense to rename the fund if/once we have a good idea.