Request for proposals: Help Open Philanthropy quantify biological risk

post by djbinder, ASB · 2022-05-12T21:28:34.575Z · EA · GW · 5 comments

Contents

  Background
  What is our goal?
  What a successful proposal might look like
  How do I apply?
  Acknowledgements
None
5 comments

Open Philanthropy is seeking proposals from those interested in contributing to a research project informing and estimating biosecurity-relevant numbers and ‘base rates’. We welcome proposals from both research organizations and individuals (at any career stage, including undergraduate and postgraduate students). The work can be structured via contract or grant.

The application deadline is June 5th.

Background

How likely is a biological catastrophe? Do the biggest risks come from states or terrorists? Accidents or intentional attacks? 

These parameters are directly decision-relevant for Open Philanthropy’s biosecurity and pandemic preparedness strategy. They determine the degree to which we prioritize biosecurity compared to other causes, and inform how we prioritize different biosecurity interventions (e.g. do we focus on lab safety to reduce accidents, or push for better DNA synthesis screening to impede terrorists?).

One way of estimating biological risk that we do not recommend is ‘threat assessment’—investigating various ways that one could cause a biological catastrophe. This approach may be valuable in certain situations, but the information hazards involved make it inherently risky. In our view, the harms outweigh the benefits in most cases.

A second, less risky approach is to abstract away most biological details and instead consider general ‘base rates’. The aim is to estimate the likelihood of a biological attack or accident using historical data and base rates of analogous scenarios, and of risk factors such as warfare or terrorism. A few examples include:

This information allows us to better estimate the probability of biological catastrophe in a variety of scenarios, with some hypothetical concrete examples described in the next section.

The biosecurity team at Open Philanthropy (primarily Damon) is currently working on developing such models. However, given the broad scope of the work, we would be keen to see additional research on this question. While we are interested in independent models that attempt to estimate the overall risk of a global biological catastrophe (GCBR), we are particularly keen on projects that simply collect huge amounts of relevant historical data, or thoroughly explore one or more sub-questions.

One aspect of this approach we find particularly exciting is that it is threat-agnostic and thus relevant across a wide range of scenarios, foreseen and unforeseen. It also helps us to think about the misuse of other dangerous transformative technologies, such as atomically precise manufacturing or AI.

We are therefore calling for applications from anyone who would be interested in spending up to the next four months (or possibly longer) helping us better understand these aspects of biorisk. We could imagine successful proposals from many different backgrounds—for example, a history undergraduate looking for a summer research project, a postdoc or superforecaster looking for part-time work, or a quantitative research organization with an entire full-time team.

What is our goal?

We are interested in supporting work that will help us better quantitatively estimate the risk of a GCBR without creating information hazards. To do this, we can imagine treating the biological details of a threat as a ‘black box’ and instead quantifying risk within hypothetical scenarios like the following:

Scenario 1 (states or big terrorist groups): In January 2025, scientists publish a paper that inadvertently provides the outlines for a biological agent that, if created and released, would kill hundreds of millions of people. Creating the pathogen would require a team of 10 PhD-level biologists working full time for 2 years, and a budget of $10 million.

Scenario 2 (small terrorist groups or lone wolf): Same as Scenario 1, but creating the pathogen requires a single PhD-level biologist working full time for 2 years, and a budget of $1 million.

For each of these scenarios, what is the ‘annual rate’ at which the biological agent is created and released (either accidentally or intentionally)?

We are interested in research proposals that aim to estimate this rate or estimate rates in similar scenarios. Proposals don’t need to do this directly, but could instead aim to quantitatively understand ‘upstream’ aspects of the world that could affect the risk of catastrophe.

The scenarios serve as a concrete litmus test for the kinds of proposals we are interested in—if the proposal wouldn’t directly help a researcher estimate the annual risk rate in these toy scenarios, it is unlikely to be of interest to us. In particular, purely qualitative approaches, such as trying to understand specific case studies in detail, or trying to understand the ideological drivers of terrorism, are unlikely to be a good fit. Similarly, very theoretical approaches, such as those with empirical unverifiable parameters or unfalsifiable elements, are also unlikely to be successful.

What a successful proposal might look like

An ideal proposal could propose things like one or more of the following:

We are interested in proposals of any length or scope, ranging from a full time 4-6+ month commitment to a small 10-hour project. In some instances, we might respond to a proposal by suggesting a closer ongoing collaboration.

How do I apply?

Applications are via this Google form, and are due on Sunday, June 5th, at 11:59 pm PDT. You’ll be required to submit:

We expect to fund between $500,000 and $2 million worth of proposals, depending on the quality and scope of proposals. In exceptional circumstances, we could expand this amount substantially.

You should hear back from us by June 26th. Please contact us at biosecurity@openphilanthropy.org if you have any further queries.

If you would like to provide anonymous or non-anonymous feedback to Open Philanthropy’s Biosecurity & Pandemic Preparedness team relevant to this project, please use this form.

Acknowledgements

Thank you to Carl Shulman for initially suggesting this research approach and providing comments. We appreciate additional comments/advice from many others, particularly Chris Bakerlee, Rocco Casagrande, and Gregory Lewis.

5 comments

Comments sorted by top scores.

comment by Tessa (tessa) · 2022-05-13T11:49:24.286Z · EA(p) · GW(p)

Some recent-ish resources that potential applicants might want to check out:

David Manheim and Gregory Lewis, High-risk human-caused pathogen exposure events from 1975-2016, data note published in August 2021.

As a way to better understand the risk of Global Catastrophic Biological Risks due to human activities, rather than natural sources, this paper reports on a dataset of 71 incidents involving either accidental or purposeful exposure to, or infection by, a highly infectious pathogenic agent.

Filippa Lentzos and Gregory D. Koblentz, Mapping Maximum Biological Containment Labs Globally, policy brief published in May 2021 part of the Global Biolabs project.

This study provides an authoritative resource that: 1) maps BSL4 labs that are planned, under construction, or in operation around the world, and 2) identifies indicators of good biosafety and biosecurity practices in the countries where the labs are located.

2021 Global Health Security Index, https://www.ghsindex.org/.

If you click through to the PDFs under each individual country profile, they have detailed information on the country's biosafety and biosecurity laws! (Example: the exact laws aren't clear from https://www.ghsindex.org/country/ukraine/ but if you click through to the "Country Score Justification Summary" PDF (https://www.ghsindex.org/wp-content/uploads/2021/12/Ukraine.pdf) it has like 100 pages of policy info.

comment by Tessa (tessa) · 2022-05-13T11:37:44.431Z · EA(p) · GW(p)

One now-inactive past project in this space that I would highlight (since I would very much like something similar to exist again) is The Sunshine Project. Quoting its (sadly very short) Wikipedia page:

The Sunshine Project worked by exposing research on biological and chemical weapons. Typically, it accessed documents under the Freedom of Information Act and other open records laws, publishing reports and encouraging action to reduce the risk of biological warfare. It tracked the construction of high containment laboratory facilities and the dual-use activities of the U.S. biodefense program.

Some more on Edward Hammond's work/methods show up in this press article on The Worrying Murkiness of Institutional Biosafety Committees:

In 2004, an activist named Edward Hammond fired up his fax machine and sent out letters to 390 institutional biosafety committees across the country. His request was simple: Show me your minutes.

...

The committees “are the cornerstone of institutional oversight of recombinant DNA research,” according to the NIH, and at many institutions, their purview includes high-security labs and research on deadly pathogens.

...

When Hammond began requesting minutes in 2004, he said, he intended to dig up information about bioweapons, not to expose cracks in biosafety oversight. But he soon found that many institutions were unwilling to hand over minutes, or were struggling to provide any record of their IBCs at all. For example, he recalled, Utah State was a hub of research into biological weapons agents. “And their biosafety committee had not met in like 10 years, or maybe ever,” Hammond said. “They didn’t have any records of it ever meeting.”

comment by NunoSempere · 2022-05-13T20:41:29.806Z · EA(p) · GW(p)

Could you change the deadline to June 5th? This seems like a potentially good fit to some readers of my forecasting newsletter, which goes out at the beginning of each month

Replies from: djbinder, djbinder
comment by djbinder · 2022-05-21T21:32:14.069Z · EA(p) · GW(p)

We have decided to extend the deadline to June 5th, if you'd still be to do advertise this in your forceasting newsletter that would be helpful!

comment by djbinder · 2022-05-17T17:33:17.334Z · EA(p) · GW(p)

Thanks for pointing this out, but unfortunately we cannot shift the submission deadline.