Project Ideas in Biosecurity for EAs

post by Davidmanheim · 2021-02-16T21:01:44.588Z · EA · GW · 13 comments


    is this list for?
  Ideas by Discipline / Subject Areas
    / Anthropology
    and Information Dispersal
    / Epidemiology

In conjunction with a group of other EA biosecurity folk, I helped brainstorm a set of projects which seem useful, and which require various backgrounds but which, as far as we know, aren't being done, or could use additional work. Many EAs have expressed interest in doing something substantive related to research in bio, but are unsure where to start - this is intended as one pathway to do so.

Note: I'm also happy to hear from people who weren't involved in our brainstorming about additional ideas - but privately! Please don't post them as comments, both because we aren't necessarily endorsing their usefulness / relevance, and because many project ideas are close to infohazards, and we don't advise people going and doing such work without (private) discussions about risks and precautions, to avoid unilateralist's curse. (For further information, see this comment. [EA(p) · GW(p)])

Ideally, this post will be updated as people publish work on these topics - let me know and I can link to Contributions, or perhaps even things to mark Done.

What is this list for?

We think that each of these is both a substantive and valuable question, which we'd like to see someone answer well. We think they are minimally info-hazardous, but still urge some degree of mindfulness about the issue. 

All of these projects should start with literature reviews - there is relevant work on all of these, and you need to know about it. Sometimes, the paper that does what we asked for exists - and if so, finding it is helpful. If not, a literature review is, by itself, often a useful post / paper, and for some of the questions, it's all that is needed. If you're not sure where to start, here's a good basic introduction [LW · GW].

If you do a literature review, and think there is more to be done, or want to publish them and aren't sure how, or want feedback, there are many researchers that can advise or help on next steps. (But only ask once you have drafts of the literature review, or better, project plans - until that point, the goal is to see if more people can do research, and have more EAs able to do this type of work on their own!)

If you're fairly confident you have a track record that shows you know how to do this type of work, many seem like good subjects for grant proposals. They would also be clear evidence that you can do research. If you're interested in academia, like getting into a graduate program is biosecurity, publishing a paper is a great idea. (If you're interested in working in EA research, since doing such projects is a good way to show people you can do that as well.) And if you're in school, and think you can make one of these into a thesis or paper, that would be great as well.

Note that many of these, or parts of them, could be something as short as a good EA forum post, but some could easily be as long as a PhD dissertation - or more. A critical part of doing research is narrowing your scope based on what you can do!

Ideas by Discipline / Subject Areas

We've split these up roughly by area of knowledge or type of work they involve. Many could be approached more than one way, but it's useful even though disciplinary boundaries are always a bit overly restrictive and imprecise. 


Sociology / Anthropology

Meta-Science and Information Dispersal


Public Policy

Biology / Epidemiology


) As an aside, this brainstorming was an interesting group activity involving a collaborative Google doc, and it led to a lot of knowledge sharing about what existed that not everyone knew about, and was very productive for me, and several others said they enjoyed it / found it useful. 


Comments sorted by top scores.

comment by eca · 2021-02-25T20:28:11.736Z · EA(p) · GW(p)

Thanks for writing this David. Its been on my todo list for a while to write down project ideas like this. I think some of these ideas are useful and worth doing, and getting those out in the open is great.

On the other hand, I think its actually pretty hard to find research which is directly good for reducing biorisk. In my experience the space of ideas which “seem maybe useful” is much larger than the set of projects which actually directly help, on more reflection. This is a general problem and not intended to be a specific critique of the ideas you shared.

I think there is a broader set of projects which are not causing direct good in the world, but are still worth doing to build skills at this type of research. I think its often better for these projects to look like speculatively good direct impact projects, rather than something wholly made up just for learning. But I think its really important to be clear when a project is in this category. Eg “I don’t think this project would be worth your time if you didn’t learn a lot from it, but I think you will so I still recommend it”.

In my opinion the tone of this post makes it sound like these ideas have been more well-vetted/ more strongly in the “directly good to do” category of projects then I assess them to be. (Speaking as a “biosecurity EA” and an individual who cares about this stuff, only, not trying to represent an organizational opinion.)

In case it is helpful for future readers to have an independent— but extremely rough/ not strongly held—assessment of these, I would put 30% in the category of “probably not worth doing even for skill building unless you have very specific goals and circumstances” 65% in the “plausibly worth doing if you think you will learn stuff” and 5% in “plausibly I would recommend someone doing this even if I thought they wouldn’t learn much” category. I think its important to share ideas for (and do!) projects in all these categories, but I would be sad if someone thought they were more widely endorsed as directly useful by “biosecurity EAs” then I believe they actually are.

Replies from: Davidmanheim
comment by Davidmanheim · 2021-02-28T17:23:29.209Z · EA(p) · GW(p)

I'd be interested in more specific private feedback on which projects you don't think would not be useful, or ideas for other things you think people with those skill sets could do that would be more useful. Cross-checking you intuitions with others would be good - for each of these projects, someone else working actively in biosecurity thought the project would be useful.  And I think that it's easy to have a narrow view of what is useful - I wouldn't have thought people would want many of these answers until they explained that they did, and often why. 

That said, if someone is interested in working directly on things that are substantially important, and has a track record for doing so, there are people who want to hire them already, and they have plenty of opportunity for collaboration with biosecurity EAs. I didn't, and don't, think that we need to provide that set of people lists of things to work on - and I would agree that this isn't the set of highest priority tasks, many of which require funding and support, or have other reasons that people cannot pick them up as side projects. This list is geared towards things that people with a diverse skill set can do as an initial step, which active biosecurity researchers have said would show them someone is capable of doing useful research, while avoiding information hazards.

Replies from: eca
comment by eca · 2021-03-02T18:52:11.192Z · EA(p) · GW(p)

Makes sense- possibly I'd change my mind about many of these after hearing the motivation. The second half of your response make me believe that we actually don't disagree that much RE a lot of the projects in here being good substantially or primarily because they could help establish a research track record or be a good learning opportunity. 

Happy to chat more about this.

comment by alexrjl · 2021-02-18T19:47:18.248Z · EA(p) · GW(p)

Thanks for writing this!

For those interested in the Sociology/Anthro side, I showed section to my partner, who's an anthropologist though not in this area. She suggested that these papers might be a helpful starting point, with the first in particular aiming to provide a framing for how the issue might be investigated anthropologically.

Biosecurity: Towards an anthropology of the contemporary

The history of biological warfare

History of biological warfare and bioterrorism

The illogic of the biological weapons taboo

comment by MichaelA · 2021-02-17T05:32:39.674Z · EA(p) · GW(p)

many project ideas are close to infohazards, and we don't advise people going and doing such work without (private) discussions about risks and precautions, to avoid unilateralist's curse.

In case any readers are unfamiliar with these terms, here are the brief EA Concepts pages on them: 

And here [? · GW] are a [EA(p) · GW(p)] few collections [EA(p) · GW(p)] of posts/sources on those topics.

How, in detail, does the cost and speed of potential solutions to a problem affect optimal levels of openness around that problem?

If I'm interpreting this question correctly, I think the following interesting paper addresses a somewhat similar question in the context of AI (and thus might be helpful to people considering addressing this question): The Offense-Defense Balance of Scientific Knowledge: Does Publishing AI Research Reduce Misuse?

(There's also some commentary on the paper here [LW · GW].)

Review of past, i.e. definitely no longer hazardous, examples of info hazards and how they’ve played out (e.g., here are 10 examples of streisand effects in history)

A useful starting point on this might be Exploring the Streisand Effect [EA · GW].

Replies from: Davidmanheim
comment by Davidmanheim · 2021-02-17T10:39:29.503Z · EA(p) · GW(p)

Thanks for this - I have added the EA Concepts links to the post, and linked to this comment for more information.

comment by JasperGeh · 2021-02-18T17:37:58.859Z · EA(p) · GW(p)

Dataset of different viral pathogens which have emerged or are discussed as novel/emerging diseases for both humans and animals, by viral realm and kingdom, Baltimore classification, transmission method, and region.

I've thought about this at some point and would be happy to chat and share my thoughts with anyone thinking about such datasets.

I'm also working on viral (and some metagenomic) sequencing for my PhD and could see myself collaborating or at least giving input to related projects on the side, should people want to pursue something along those lines.

comment by MichaelA · 2021-02-17T05:18:58.877Z · EA(p) · GW(p)

Thanks for compiling this - this seems like a helpful resource! I've added it to A central directory for open research questions [EA · GW]. 

Personally, I find myself most interested in the questions in the sociology / anthropology section, and I've made a note to think later about whether it might be worth me doing some work on one or more of them. 

Readers of this post might also be interested in the "biorisk and biotechnology" cluster of my post (for Convergence Analysis) on Crucial questions for longtermists [EA · GW]. (The questions there tend to be more zoomed-out and less guided by actual detailed knowledge of biosecurity issues than the questions in your post. I think they also touch on some other topics and provide a complementary perspective/framing.)

comment by MichaelA · 2021-02-27T02:10:28.242Z · EA(p) · GW(p)

I just saw this paper from researchers from CSER and elsewhere: 80 Questions for UK Biological Security. I haven't read it, but imagine it might be useful to some of the people for whom this post would be useful, so I figured I'd mention it here.

Here's the abstract:

Multiple national and international trends and drivers are radically changing what biological security means for the United Kingdom (UK). New technologies present novel opportunities and challenges, and globalisation has created new pathways and increased the speed, volume and routes by which organisms can spread. The UK Biological Security Strategy (2018) acknowledges the importance of research on biological security in the UK. Given the breadth of potential research, a targeted agenda identifying the questions most critical to effective and coordinated progress in different disciplines of biological security is required. We used expert elicitation to generate 80 policy-relevant research questions considered by participants to have the greatest impact on UK biological security. Drawing on a collaboratively-developed set of 450 questions, proposed by 41 experts from academia, industry and the UK government (consulting 168 additional experts) we subdivided the final 80 questions into six categories: bioengineering; communication and behaviour; disease threats (including pandemics); governance and policy; invasive alien species; and securing biological materials and securing against misuse. Initially, the questions were ranked through a voting process and then reduced and refined to 80 during a one-day workshop with 35 participants from a variety of disciplines. Consistently emerging themes included: the nature of current and potential biological security threats, the efficacy of existing management actions, and the most appropriate future options. The resulting questions offer a research agenda for biological security in the UK that can assist the targeting of research resources and inform the implementation of the UK Biological Security Strategy. These questions include research that could aid with the mitigation of Covid-19, and preparation for the next pandemic. We hope that our structured and rigorous approach to creating a biological security research agenda will be replicated in other countries and regions. The world, not just the UK, is in need of a thoughtful approach to directing biological security research to tackle the emerging issues.

comment by Simon_Grimm · 2021-02-27T09:22:52.963Z · EA(p) · GW(p)

Thanks for this write-up. Concerning this point:

Quantitative investigation of tech capabilities required for broad environmental nucleic acid surveillance to be useful

This article provides a good introduction to current challenges within genomic pathogen surveillance: Ten recommendations for supporting open pathogen genomic analysis in public health

Replies from: Davidmanheim
comment by Davidmanheim · 2021-02-28T17:28:20.717Z · EA(p) · GW(p)

Thanks. Having looked through the paper briefly, it seems like it pays no attention to any hazards or misuse, and doesn't actually address the challenges of developing or  supporting infrastructure for analysis, especially the things they suggest which relate to PII.

comment by MichaelA · 2021-02-17T05:37:52.186Z · EA(p) · GW(p)

If you do a literature review, and think there is more to be done, or want to publish them and aren't sure how, or want feedback, there are many researchers that can advise or help on next steps. (But only ask once you have drafts of the literature review, or better, project plans - until that point, the goal is to see if more people can do research, and have more EAs able to do this type of work on their own!)

Do you have a specific suggestion for which researchers people should contact, or how they should find out? E.g., contact you and let you direct them?

And are you indeed suggesting that people should not reach out for advice or help on next steps until they've developed at least drafts of a lit review and/or project plans? That seems surprising to me. It seems to me that:

  • a sizeable fraction of people who are considering pursuing these project ideas would probably benefit from reaching out to researchers for advice at an earlier stage (e.g. to outline and get feedback on their current skills and interests and their vague ideas on which directions they could head in)
  • EA researchers are often quite happy to help at that stage
  • (This has been my experience to date, both as an advice-seeker and an advice-giver, though not specifically in relation to biosecurity stuff.)

(I don't actually need answers to these questions myself. I just feel like these questions might be useful for some other readers, since I think/hope readers might in future use this post to help them decide whether and how to try out / further pursue biosecurity-related research.)

Replies from: Davidmanheim
comment by Davidmanheim · 2021-02-17T10:36:01.705Z · EA(p) · GW(p)

For earlier stage discussions, I agree that some people are interested in providing general career guidance, and perhaps that means suggesting specific projects from this list - but that's different than requesting help getting started on a specific project. 

A key part of doing useful research is thinking about a question and figuring out what to do to investigate, and while I and others could flesh these out into project outlines ourselves, a large part of the goal in posting this is to let others show their capabilities for doing so themselves. Moreover, people who aren't able to at least start such a project are unlikely to be successful as researchers - and doing the initial steps is intended to be a gauge of both commitment and ability.

Providing guidance on how to start and work on these projects requires a significant investment of time - and I don't think it is fair to bother others with volunteers who haven't shown they are interested and at least somewhat capable. I and others are happy to provide guidance if researchers interested in these problems are stuck, but some work beyond "I think I'd like to do this"  is a prerequisite for getting feedback. 

With that said, I am happy to be a contact point for coordinating any work people are interested in doing, and I can put you in touch with others who are interested in the specific projects.