Centre for the Study of Existential Risk Six Month Report: November 2018 - April 2019

post by HaydnBelfield · 2019-05-01T15:34:20.425Z · score: 10 (13 votes) · EA · GW · 16 comments

Contents

    Contents:
  1. Overview
    2. Policy Engagement:
    3. Academic and Industry Engagement:
    4. Public Engagement:
    5. Recruitment and research team
      New Postdoctoral Research Associates:
      Visiting researchers:
    6. Expert Workshops and Public Events:
    7. Upcoming activities
    8. Publications
17 comments

We have just prepared a Six Month Report for our Management Board. This is a public version of that Report.  We send short monthly updates in our newsletter – subscribe here.

READ AS PDF

Contents:

  1. Overview
  2. Policy and Industry Engagement
  3. Academic Engagement
  4. Public Engagement
  5. Recruitment and research team
  6. Expert Workshops and Public Lectures
  7. Upcoming activities
  8. Publications

1. Overview

The Centre for the Study of Existential Risk (CSER) is an interdisciplinary research centre within the University of Cambridge dedicated to the study and mitigation of risks that could lead to civilizational collapse or human extinction. We study existential risk, develop collaborative strategies to reduce them, and foster a global community of academics, technologists and policy-makers working to tackle these risks. Our research focuses on Global Catastrophic Biological Risks, Extreme Risks and the Global Environment, Risks from Artificial Intelligence, and Managing Extreme Technological Risks.

Our last Management Board Report was in October 2018. Over the last five months, we have continued to advance existential risk research and grow the field. Highlights include:

2. Policy Engagement:

We have had the opportunity to speak directly with policymakers and institutions across the world who are grappling with the difficult and novel challenge of how to unlock the socially beneficial aspects of new technologies while mitigating their risks. Through advice and discussions, we have the opportunity to reframe the policy debate and to hopefully shape the trajectory of these technologies themselves.

3. Academic and Industry Engagement:

As an interdisciplinary research centre within Cambridge University, we seek to grow the academic field of existential risk research, so that it receives the rigorous and detailed attention it deserves. Researchers also continued their extensive and deep collaboration with industry. Extending our links improves our research by exposing us to the cutting edge of industrial R&D, and helps to nudge powerful companies towards more responsible practices.

4. Public Engagement:

We are able to reach far more people with our research online:
14,000 website visitors over the last 90 days.
6,602 newsletter subscribers, up from 4,863 in Oct 2016.
6,343 Twitter followers.
2,208 Facebook followers.

5. Recruitment and research team

New Postdoctoral Research Associates:

Visiting researchers:

6. Expert Workshops and Public Events:

7. Upcoming activities

Three more books will be published this year:

Upcoming events:

Timing to be confirmed:

8. Publications

16 comments

Comments sorted by top scores.

comment by SiebeRozendal · 2019-05-01T18:47:22.237Z · score: 31 (11 votes) · EA · GW

What is the relevance of "the link between biodiversity and economic growth" to existential risk? It is not immediately obvious to me.

comment by HaydnBelfield · 2019-05-01T21:46:54.942Z · score: 6 (11 votes) · EA · GW

Thanks for the question. Biodiversity loss and associated catastrophic ecosystem shifts are a contributor to existential risk. Partha's review may influence UK and international policy.

See:

We also have further publications forthcoming on the link between biodiversity and existential risk.

comment by Halstead · 2019-05-02T16:28:49.833Z · score: 46 (13 votes) · EA · GW

Can you explain what the mechanism is whereby biodiversity loss creates existential risk? And if biodiversity loss is an existential risk, how big a risk is it? Should 80k be getting people to go into conservation science or not?

There are independent reasons to think that the risk is negligible. Firstly, according to wikipedia, during the Eocene period ~65m years ago, there were thousands fewer genera than today. We have made ~1% of species extinct, and we would have to continue at current rates of species extinctions for at least 200 years to return to Eocene levels of biodiversity. And yet, even though significantly warmer than today, the Eocene marked the dawn of thousands of new species. So, why would we expect the world 200 years hence to be inhospitable to humans if it wasn't inhospitable for all of the species emerging in the Eocene, who are/were significantly less numerous than humans and significantly less capable of a rational response to problems?

Secondly, as far as I am aware, evidence for pressure-induced non-linear ecosystem shifts is very limited. This is true for a range of ecosystems. Linear ecosystem damage seems to be the norm. If so, this leaves more scope for learning about the costs of our damage to ecosystems and correcting any damage we have done.

Thirdly, ecosystem services are overwhelmingly a function of the relations within local ecosystems, rather than of global trends in biodiversity. Upon discovering Hawaii, the Polynesians eliminated so many species that global decadal extinction rates would have been exceptional. This has next to no bearing on ecosystem services outside Hawaii. Humanity is an intelligent species and will be able to see if other regions are suffering from biodiversity loss and make adjustments accordingly. Why would all regions be so stupid as to ignore lessons from elsewhere? Also, is biodiversity actually decreasing in the rich world? I know forest cover is increasing in many places. Population is set to decline in many rich countries in the near future, and environmental impact per person is declining on many metrics.

I also find it surprising that you cite the Kareiva and Carranza paper in support of your claims, for this paper in fact directly contradicts them:

"The interesting question is whether any of the planetary thresholds other than CO2 could also portend existential risks. Here the answer is not clear. One boundary often mentioned as a concern for the fate of global civilization is biodiversity (Ehrlich & Ehrlich, 2012), with the proposed safety threshold being a loss of greater than 0.001% per year (Rockström et al., 2009). There is little evidence that this particular 0.001% annual loss is a threshold—and it is hard to imagine any data that would allow one to identify where the threshold was (Brook, Ellis, Perring, Mackay, & Blomqvist, 2013; Lenton & Williams, 2013). A better question is whether one can imagine any scenario by which the loss of too many species leads to the collapse of societies and environmental disasters, even though one cannot know the absolute number of extinctions that would be required to create this dystopia.

While there are data that relate local reductions in species richness to altered ecosystem function, these results do not point to substantial existential risks. The data are small-scale experiments in which plant productivity, or nutrient retention is reduced as species numbers decline locally (Vellend, 2017), or are local observations of increased variability in fisheries yield when stock diversity is lost (Schindler et al., 2010). Those are not existential risks. To make the link even more tenuous, there is little evidence that biodiversity is even declining at local scales (Vellend et al., 2013, Vellend et al., 2017). Total planetary biodiversity may be in decline, but local and regional biodiversity is often staying the same because species from elsewhere replace local losses, albeit homogenizing the world in the process. Although the majority of conservation scientists are likely to flinch at this conclusion, there is growing skepticism regarding the strength of evidence linking trends in biodiversity loss to an existential risk for humans (Maier, 2012; Vellend, 2014). Obviously if all biodiversity disappeared civilization would end—but no one is forecasting the loss of all species. It seems plausible that the loss of 90% of the world’s species could also be apocalyptic, but not one is predicting that degree of biodiversity loss either. Tragic, but plausible is the possibility of our planet suffering a loss of as many as half of its species. If global biodiversity were halved, but at the same time locally the number of species stayed relatively stable, what would be the mechanism for an end-of-civilization or even end of human prosperityscenario? Extinctions and biodiversity loss are ethical and spiritual losses, but perhaps not an existential risk."

comment by Sean_o_h · 2019-05-02T19:49:20.389Z · score: 16 (8 votes) · EA · GW

Hi John, thanks for these detailed points and considerations. I'd like to add a few comments of my own (disclosure that I'm co-Director of CSER, although quite a bit of what's below represents my individual opinion, as flagged).

1) I should note that there isn't a 'CSER position' re: biodiversity loss directly causing an existential risk to humanity, or the extent to which it's a cause of concern as a contributing factor. I don't believe I know of anyone holding the former stronger view on current evidence, and the extent to which different people weight the latter differs both between researchers and between advisers.

2) While I note that I'm not a domain expert on biodiversity loss, my own individual view leans towards the Kareiva & Carranza you quote above (from a paper presented at one of our conferences). I'd note that other experts appear to disagree (e.g. https://www.theguardian.com/environment/2018/nov/03/stop-biodiversity-loss-or-we-could-face-our-own-extinction-warns-un) though I'm disinclined to weight strong statements from agency heads in public media as strong and reliable evidence.

3) Re: "is biodiversity loss declining in rich countries?" Recent reports do indicate that biodiversity is continuing to decline, and ecosystems continue to be under threat in Europe, despite this being a (comparatively) rich region, and despite strategies intended to combat this - e.g. see

https://www.eea.europa.eu/soer-2015/europe/biodiversity

https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52015DC0478

4) Even if not an existential risk, these reports indicate a significant negative impact on the global economy from global biodiversity loss "Policy inaction and failure to halt the loss of global biodiversity could result in annual losses in ecosystem services equivalent to 7 % of world GDP, with the greatest impacts being felt by the poorest nations and the rural poor. "

5) There are a lot of matters that remain unclear, such as the interrelationship and possible feedback loops between climate change, biodiversity loss, increased resource use etc, that in my view would be useful to understand better in order to better understand the effect of biodiversity loss on human civilisation, and how it fits into the bigger picture of global catastrophic risks to humanity in the coming decades. As I understand it, it remains unclear what the plausible worst case scenario is.

6) My own (non-expert) view is that it's worth it for the GCR community not to ignore global biodiversity loss, due to the dynamic and unprecedented-in-human-history nature of the change, the interaction with other pressures with other potential GCR significance, and plausible reasons to think this may have large negative consequences for the planet and human civilisation in its current form (which can have destabilising consequences). To your question to Haydn, I don't think 80K should be recommending it as a top cause area based on current evidence, but I may update on this in future years in light of further evidence. At CSER it's a small part of our current portfolio and resource use, which I think is appropriate (as indicated by the fraction on the report above it takes up; also worth noting our leading work is currently being done by a non-grant-supported professor). It is of course a particularly influential part in various regards, given that Prof Dasgupta is in an unusually influential position and can achieve a lot of good on a topic of global significance (with potentially significant effects on global human well-being and productivity as noted above).

7) >Secondly, as far as I am aware, evidence for pressure-induced non-linear ecosystem shifts is very limited. This is true for a range of ecosystems.

My understanding is that this is correct. A project currently being written up was looking to review the evidence on this in order to better understand how concerned to be about this possibility, although it's proven difficult to gather sufficient evidence from the literature. I'll be better-placed to know what to conclude from this once write up is complete.

Apologies that these comments aren't in correlated order with your post; I'll go back and reorder if required. Again, I'd highlight that others associated with CSER may hold stronger (and more expert) views than I on this topic.

comment by Halstead · 2019-05-03T08:26:18.961Z · score: 10 (3 votes) · EA · GW

3. I have a sceptical prior against EU studies of scientific issues because the EU has taken an anti-science stance on many issues under pressure from the environmental movement - see e.g. the effective prohibition of GMOs. The fact that the report you cite advocates for increased organic farming adds weight to my scepticism. The report also says that the estimate of the economic costs is extremely uncertain and potentially a massive overestimate.

4. There are many things in the world that impose substantial economic costs, including inefficienct taxation, labour market regulation, failure to invest in R&D, etc. While they may indeed create economic costs, I fail to see the connection to existential risk.

5. While it is a small part of your portfolio, there is limited political attention for existential risk, and if CSER does start advocating for the view that biodiversity loss deserves serious consideration as a factor relevant to existential risk, that comes at a cost. In this case, the fact that Partha Dasgupta is an influential person is a negative because he risks distracting policymakers from the genuine risks

comment by Sean_o_h · 2019-05-03T08:51:02.487Z · score: 2 (2 votes) · EA · GW

Thanks John. With apologies for brevity: I don't think I'd agree with such broad-strokes scepticism of EU scientific studies on environment, but this is a topic for a longer conversation. Ditto (4).

Re: 5, I don't expect this to be the framing that Partha adopts in the review in question; rather I expect it will be in line with the kinds of analysis and framings he has adopted in his work in this space in the past years (on the basis of which he was chosen for this appointment). Thanks!

comment by SiebeRozendal · 2019-05-03T16:38:59.441Z · score: 18 (5 votes) · EA · GW

Re 5: To be honest, I doubt that his framing matters much. Whether it's "influential person says Y should receive attention" or "influential person says Y should receive attention with a lot of caveats" it's still a distraction if we think Y is not nearly as relevant as X.

I think this point to a wider issue about risk communication and advocacy: should the x-risk community:

1) advocate for many approaches to x-risk and be opportunistic in where policy-makers are responsive, or

2) advocate for addressing the biggest risks only and bullishly pursue only opportunities that address these biggest risks.

This seems to depend on 'how widely is x-risk distributed over various risk factors?' and different research organizations seem to hold different opinions. Is CSER's view that x-risk is widely distributed or narrowly?

comment by Sean_o_h · 2019-05-04T13:59:37.048Z · score: 17 (6 votes) · EA · GW

>Re 5: To be honest, I doubt that his framing matters much. Whether it's "influential person says Y should receive attention" or "influential person says Y should receive attention with a lot of caveats" it's still a distraction if we think Y is not nearly as relevant as X.

From my experience of engaging with policymakers on Xrisk/GCR, I disagree with this way of looking at things (and to an extent John's related concerns). If Partha was directly pushing biodiversity loss as a direct existential risk to humanity needing policy action, without evidence for this, then yes I would have concerns about this. But that's not what's happening here. At most, some 'potential worst case scenarios' might be surfaced, and referred to centres like ours for further research to support or rule out.

A few points:

1) I think it's wrong to view this as a zero sum game. There's a huge, huge space for policymakers to care more about GCR, Xrisk, and the long-term future than they currently do. Working with them on a global risk-relevant topic they're already planning to work on (biodiversity and economic growth), as Partha is doing, is not going to result in the space that could be taken up with Xrisk concerns being occupied.

2) What we have here is a leading scholar (with a background specifically in economics and in recent years, biodiversity/sustainability) working in a high-profile fashion on a global risk-relevant topic (biodiversity loss and economics), who also has strong links to an existential risk research centre. This establishes useful links; it demonstrates that scholars associated with existential risk (a flaky-seeming topic not so long ago, and still in some circles) are people who do good work and are useful and trustworthy for governments on risks already within their 'attention' overton window; it's helpful for legitimacy and reputation of existential risk research (e.g. through these links, interactions, and reputable work on related topics, helping to nudge existential risks into the overton window of risks that policymakers take seriously and take active government action on.)

More broadly, and to your later points:

Working on these sorts of processes is also an effective way of understanding how governance and policy around major risk works, and developing the skillset and positioning needed to engage more effectively around other risks (e.g. existential).

We don't know all the correct actions to take to prevent existential risks right now. In some cases (i) because the xrisks will come to light in future; (ii) in some cases because we know the problem but don't yet know how to solve; (iii) in some cases because we have a sense of the solution but not a good enough sense of how to action. For all these things, doing some engagement in policy processes where we can work to mitigate global risks currently within the policy overton window can be useful.

I do think the Xrisk community needs 'purists', and there will be points at which the community will need to undertake a hard prioritisation action on a particular xrisk with government. But most within the community would agree it's not the time with transformative AI; it's not the time with nano; there's disagreement over whether it is the time with nuclear. With bio, a productive approach is expanding the overton window of risks within current biosecurity and biosafety, which is made easier by being clearly competent and useful within these broader domains.

What it is time for is internally doing the research to develop answers. Externally and with policy communities, developing the expertise to engage with the mechanics of the world, the networks and reputation to be effective, embedding the foresight and risk-scanning/response mechanisms that will allow governments to be more responsive, and so forth. Some of that involves engaging with a wider range of global (but not necessarily existential) risk issues. (As well as other indirect work: e.g. the AI safety/policy community not just working on the control problem and the deployment problem, but also getting into position in a wide range of other ways that often involve broader processes or non-existential risk issues).

To your final question, my own individual view is that mitigating xrisk will involve a small number of big opportunities/actions at the right times, underpinned and made possible by a large number of smaller and more widely distributed ones.

Apologies that I'm now out of time for further engagement online due to other deadlines.

comment by HaydnBelfield · 2019-05-02T17:28:04.497Z · score: 2 (2 votes) · EA · GW

Hi John, thanks for the very detailed response. My claim was that ecosystem shift is a "contributor" to existential risk - my claim is that it should be examined to assess the extent to which it is a "risk factor" that increases other risks, one of a set of causes that may overwhelm societal resilience, and a mechanism by which other risks cause damage.

As I said in the first link, "humanity relies on ecosystems to provide ecosystem services, such as food, water, and energy. Sudden catastrophic ecosystem shifts could pose equally catastrophic consequences to human societies. Indeed environmental changes are associated with many historical cases of societal ‘collapses’; though the likelihood of occurrence of such events and the extent of their socioeconomic consequences remains uncertain."

I can't respond to your comment at the length it deserves, but we will be publishing papers on the potential link between ecosystem shifts and existential risk in the future, and I hope that they will address some of your points.

I'll email you with some related stuff.

comment by Halstead · 2019-05-03T08:05:19.102Z · score: 10 (6 votes) · EA · GW

There are lots of risk factors for societal resilience to catastrophes, including all contemporary political and economic problems. The key question is how much of a risk they are and I have yet to see any evidence that biodiversity loss is among the top ones.

comment by vaidehi_agarwalla · 2019-05-01T19:23:18.364Z · score: 20 (10 votes) · EA · GW

It isn't clear to me what the relationship between the business school ranking paper to x-risk is, what is the goal of such research?

comment by HaydnBelfield · 2019-05-01T21:49:41.887Z · score: -3 (9 votes) · EA · GW

Thanks for the question. Climate change is a contributor to existential risk. Changing what business schools teach (specifically to include sustainability) might change the behaviour of the next generation of business leaders.

See:

We also have further publications forthcoming on the link between climate change and existential risk.

comment by SiebeRozendal · 2019-05-03T16:21:12.626Z · score: 30 (7 votes) · EA · GW

This seems like a very long expected causal chain, and therefore - unless each link is specifically supported by evidence - unlikely to produce much effect compared to other approaches. It seems to assume:

1) Climate change is a relatively large x-risk factor (I interpreted the presentation I saw of your forthcoming article as claiming that "climate change is a non-negligible risk factor, but not a relatively large one").

2) Improving sustainability of businesses and business leaders is a relatively effective way of addressing climate change (possibly, but there are many alternatives)

3) Increasing the amount of sustainability in business school programs will improve the sustainability of business and business leaders (There seem more direct ways of influencing business leaders; Examples: what about corporate campaigns but focused on sustainability? What about carbon taxes?)

4) Affecting business rankings will affect the curriculum (Yes, this seems to happen)

It might be the case that this was an opportunity that passed by Ellen Quiqley and was low-effort to give input on. But I'm afraid this was not a great use of time, and furthermore I'm afraid this validates the - for lack of a better term - "good-by-association fallacy":

Cause Y is important.
Intervention A addresses cause Y.
Therefore, intervention A is a good use of resources.

I think this fallacy is a harmful meme that poses a risk to the EA and x-risk brand, because it's very bad prioritization.

comment by Sean_o_h · 2019-05-04T14:15:20.985Z · score: 10 (4 votes) · EA · GW

Thank you. Some specific info: Ellen Quigley joined as (part)-salaried at CSER in January 2019 (previously she was an external collaborator). The report was published in January 2019. It was conducted and mostly completed as part of a Judge Business School project in 2018. I was happy for CSER to co-brand as (a) it's a good piece of work (b) being published by someone on staff (and where others provided some previous input) with (c) a well-thought out strategic aim, with good reasons to think it would be effective and timely in its aims from people with a lot of expertise in the topic (d) on a topic within our remit (climate/sustainability) and (e) offered various potential networking and reputational opportunities.

Since the report launch, Ellen has focused on other projects - the report has high value (by usual postdoctoral project standards) followup opportunities, but there are other projects of higher priority from a GCR/Xrisk perspective. Our current thinking is that if non-fungible-for Xrisk funding becomes available, Ellen may supervise a postdoc/research assistant in designing/actioning followups. Ellen has also accepted a more direct action-focused part-appointment (advising on the university of cambridge's investment and shareholder engagement strategy around climate change (https://www.staff.admin.cam.ac.uk/general-news/two-environmental-appointments-at-the-university) so her research time is more limited.

More broadly, there are a lot of reasons why centres will sometimes engage in projects with indirect impacts or longer causal chains that don't boil down to 'failure to understand basic prioritisation for impact'. These include: 1) good intellectual or evidence-based reasons to have confidence that indirect approaches/longer causal chain-based approaches are likely to be effective, either in of themselves or as part of a suite of activities. (2) Value of these projects in establishing strong networks and credibility with bodies likely to be relevant for broader Xrisk mitigation (3) developing the ability and skillset to engage with the machinery of the world in different regards.

It will sometimes be affected by external constraints (e.g. funding stipulations - not every organisation has full funding from fully xrisk-aligned funders - or need for researchers to establish/maintain reputation and credibility in their 'home domains' in order to remain effective in the roles they play in Xrisk research). This is likely particularly true in academic institutions.

I would expect that with most xrisk organisations, particularly those with an active engagement with other research communities, policy bodies etc, there will be a suite of outputs where some are very obviously and directly relevant to xrisk, and where others are less direct or obvious but have good reason within an overall suite of activities.

My apologies in advance that I don't have time to engage further due to other deadlines.

comment by SiebeRozendal · 2019-05-07T19:08:44.571Z · score: 11 (4 votes) · EA · GW

Thanks for the elaborate response Seán. It's valuable for the EA community to understand the internal considerations of x-risk organization, and I don't want to disincentivize organisations from publishing updates like these on the forum.

Just to be clear: I was not accusing CSER of 'failure to understand basic prioritisation for impact'. I meant to say that it's hard for outsiders to evaluate the reasons why an organisation chooses to pursue a certain project. When pure/direct x-risk related projects are reported together with these indirect projects, that can reinforce the 'good-by-association fallacy' in the outsiders.

I would expect that with most xrisk organisations, particularly those with an active engagement with other research communities, policy bodies etc, there will be a suite of outputs where some are very obviously and directly relevant to xrisk, and where others are less direct or obvious but have good reason within an overall suite of activities.

I think you're right about that, although this does not necessarily mean that the current portfolio equals this 'realistic ideal' portfolio. I'm also wondering how much of the indirectness is necessary to make progress. A higher degree of indirect projects probably makes x-risk organization mainstream quicker, but at a larger risk that 'existential risk' becomes a diluted term and co-opted by other organizations.

comment by vaidehi_agarwalla · 2019-05-02T04:22:17.657Z · score: 4 (4 votes) · EA · GW

Hi Haydn, thanks for the links, looking forward to learning more about CSER's views on this. I wasn't aware that CSER was actively doing projects to promote sustainability and climate change.