Do you include in "People working specifically on AGI" people working on AI safety, or just capabilities?
Just capabilities (in other words, people working to create AGI), although I think the safety/capabilities distinction is less clear-cut outside of a few dedicated safety orgs like MIRI.
"bullish" in the sense of "thinking transformative AI (TAI) is coming soon"
what do you mean by "experts not working on AGI"?
AI people who aren't explicitly thinking of AGI when they do their research (I think this correctly describes well over 90% of ML researchers at Google Brain, for example).
Why say "even"
Because it might be surprising (to people asking or reading this question who are imagining long timelines) to see timelines as short as the ones AI experts believe, so the second point is qualifying that AGI experts believe it's even shorter.
In general it looks like my language choice was more ambiguous than desirable so I'll edit my answer to be clearer!
“I wish it need not have happened in my time," said Frodo. "So do I," said Gandalf, "and so do all who live to see such times. But that is not for them to decide. All we have to decide is what to do with the time that is given us.”
I think there's some evidence that Metaculus, while a group of fairly smart and well-informed people, are nowhere near as knowledgeable as a fairly informed EA (perhaps including a typical user of this forum?) for the specific questions around existential and global catastrophic risks.
One example I can point to is that for this question on climate change and GCR before 2100 (that has been around since October 2018), a single not-very-informative comment from me was enough to change the community median from 24% to 10%. This suggests to me that Metaculus users did not previously have strong evidence or careful reasoning on this question, or perhaps GCR-related thinking in general.
Now you might think that actual superforecasters are better, but based on the comments given so far for COVID-19, I'm unimpressed. In particular the selected comments point to use of reference classes that EAs and avid Metaculus users have known to be flawed for over a week before the report came out (eg, using China's low deaths as evidence that this can be easily replicated in other countries as the default scenario).
Now COVID-19 is not an existential risk or GCR, but it is an "out of distribution" problem showing clear and fast exponential growth that seems unusual for most questions superforecasters are known to excel at.
Do you think it makes sense for EAs to treat global health and economic development as the same cause area, given that they seem to be two somewhat separate fields with different metrics, different theories of change, different institutions etc?
(I may not be formulating this question correctly).
Do you have thoughts on whether there are public health funding opportunities (particularly those aimed at the developing world) in COVID-19 that are plausibly on par with Givewell's current top charities?
This article suggests that China's fine with shipping out masks to other countries, and also they 12x'd production in the last month or so.
"Li Xingqian, director of the foreign trade department at the Ministry of Commerce, said China had not banned the export of masks or related materials during the coronavirus epidemic, despite other countries imposing such limits.
“Masks are freely traded products … companies can trade them in line with market principles,” Li said.
"His comments pointed to Beijing’s growing confidence in its mask production capacity, after severe shortages in late January and early February.
"As of Saturday, China’s daily output of masks, including surgical masks and medical N95 masks, was 116 million units, or 12 times the output at the start of February, the country’s top economic planning agency said on Monday."
The prediction platform Metaculus is interested in a part or full-time project lead for their new pandemic site.
You get to administer and choose a lot of questions for Metaculus forecasters to predict things on, as well as decide things like question resolution.
I think this is a plausibly useful thing to do because expert forecasts are all over the place, so if you figure out a way to help shepherd/guide amateurs to forecast well, I see a plausible world where Metaculus increasingly gets used to help with more decisions (a la Good Judgement project).
I also generally think it might be valuable to have more superforecasters and similar people in our community.
I'm very excited about this! I think if I had something like this in high school (similar formatting and tone, but a lot more content), I'd be a lot less lost then and would have found my way to doing useful stuff/thinking in an utilitarian way several years earlier!
Cute theoretical argument for #flattenthecurve at any point in the distribution
What is #flattenthecurve?
The primary theory behind #flattenthecurve is assuming that everybody who will get COVID-19 will eventually get it anyway...is there anything else you can do?
It turns out it’s very valuable to
Delay the spread so that a) the peak of the epidemic spread is lower (#flattenthecurve)
Also to give public health professionals, healthcare systems, etc more time to respond (see diagram below)
A tertiary benefit is that ~unconstrained disease incidence (until it gets to herd immunity levels) is not guaranteed, with enough time to respond, aggressive public health measures (like done in Wuhan, Japan, South Korea etc) can arrest the disease at well below herd immunity levels
Why should you implement #flattenthecurve
If you haven’t been living under a rock, you’ll know that COVID-19 is a big deal
We have nowhere near the number of respirators, ICU beds, etc, for the peak of uncontrolled transmission (Wuhan ran out of ICU beds, and they literally built a dozen hospitals in a week, a feat Western governments may have trouble doing)
When should you implement #flattenthecurve policies?
A lot of people are waiting for specific “fire alarms” (eg, public health authorities sounding the bell, the WHO calling it a pandemic, X cases in a city) before they start taking measures.
I think this is wrong.
The core (cute) theoretical argument I have is that if you think #flattenthecurve is at all worth doing at any time, as long as you're confident you are on the growth side of the exponential growth curve, slowing the doubling time from X days (say) to 2X days is good for #flattenthecurve and public health perspective no matter where you are on the curve.
Okay, let’s consider a few stricter versions of the problem
Exponential growth guaranteed + all of society
One way to imagine this is if #society all implemented your policy (because of some Kantian or timeless decision theory sense, say)
Suppose you are only willing to take measures for Y weeks, and for whatever reason the measures are only strong enough to slow down the virus's spread rather than reverse the curve.
if the doubling rate is previously 3 days and everybody doing this can push it down to 8 days (or push it up to 2 days), then it's roughly equally good (bad) no matter when on the curve you do those measures.
Exponential growth NOT guaranteed + all of society
Next, relax the assumption of exponential growth being guaranteed and assume that measures are strong enough to reverse the curve of exponential growth (as happened in China, South Korea, Japan)
I think you get the same effect where the cost of X weeks of your measures should be the same no matter where you are on the curve, plus now you got rid of the disease (with the added benefit that if you initiate your measures early, less people die/get sick directly and it's easier to track new cases)
A downside is that a successful containment strategy means you get less moral credit/people will accuse you of fearmongering, etc.
NOT all of society
Of course, as a private actor you can’t affect all of society. Realistically (if you push hard), your actions will be correlated with only a small percentage of society. So relax the assumption that everybody does it, and assume only a few % of people will do the same actions as you.
But I think for #flattenthecurve purposes, the same arguments still roughly hold.
Now you’re just (eg) slowing the growth rate from 3 days to 3.05 days instead of 3 days to 8 days.
But the costs are ~ linear to the number of people who implement #flattenthecurve policies, and the benefits are still invariant to timing.
How do we know that we are on the growth side of the exponential/S curve?
Testing seems to lag actual cases a lot.
My claim is that approximately if your city has at least one confirmed or strongly suspected case of community transmission, you’re almost certainly on the exponential trajectory
Aren’t most other people’s actions different depending on where you are on the curve?
Sure, so maybe some mitigation actions are more effective depending on other people’s actions (eg, refusing to do handshakes may be more effective when not everybody has hand sanitizer than when everybody regularly uses hand sanitizer, for example)
I think the general argument is still the same however
Over the course of an epidemic, wouldn’t the different actions result in different R0 and doubling times, so you’re you're then doing distancing or whatever from a different base?
Okay, I think this is the best theoretical argument against the clean exponential curve stuff.
I still think it’s not obvious that you should do more #flattenthecurve policies later on, if anything this pushes you to doing it earlier
If you think #flattenthecurve is worthwhile to do at all (which I did not argue for much here, but is extensively argued elsewhere), it’s at least as good to do it now as it is to do it later, and plausibly better to do soon rather than later.
Among long-termist EAs, I think there's a lot of healthy disagreement about the value-loading (what utilitarianism.net calls "theories of welfare") within utilitarianism. Ie, should we aim to maximize positive sentient experiences, should we aim to minimize negative sentient experiences, or should we focus on complexity of value and assume that the value loading may be very complicated and/or include things like justice, honor, nature, etc?
My impression is that the Oxford crowd (like Will MacAskill and the FHI people) are most gung ho about the total view and the simplicity needed to say pleasure good, suffering bad. It helps that past thinkers with this normative position have a solid track record.
I think Brian Tomasik has a lot of followers in continental Europe, and a reasonable fraction of them are in the negative(-leaning) crowd. Their pitch is something like "in most normal non-convoluted circumstances, no amount of pleasure or other positive moral goods can justify a single instance of truly extreme suffering."
My vague understanding is that Bay Area rationalist EAs (especially people in the MIRI camp) generally believe strongly in the complexity of value. A simple version of their pitch might be something like "if you could push a pleasure button to wirehead yourself forever, would you do it? If not, why are you so confident about it being the right recourse for humanity?"
Of the three views, I get the impression that the "Oxford view" gets presented the most for various reasons, including that they are the best at PR, especially in English speaking countries.
In general, a lot of EAs in all three camps believe something like "morality is hard, man, and we should try to avoid locking in any definitive normative results until after the singularity." This may also entail a period of time (maybe thousands of years) on Earth to think through things, possibly with the help of AGI or other technologies, before we commit to spreading throughout the stars.
I broadly agree with this stance, though I suspect the reflection is going to be mostly used by our better and wiser selves on settling details/nuances within total (mostly hedonic) utilitarianism rather than discover (or select) some majorly different normative theory.
We may need to operationalize "top EA cause area" more precisely but I would concur with Buck/also bet money odds that <20% of a reasonable random sample of EAs will not answer a question like "in 2025, will psychedelics normalization be a top 5 priority for EAs?" in the affirmative.
3. We can flatten the disease spread curve enough that the peak hospital usage is less bad. 4. The disease mysteriously dies in the summer like every other flu or coronavirus. 5. Vaccines or good protocols are developed. 6. This gets under control due to a concerted public health efforts (as appears to have happened in China)
This has made me more confident since I wrote the document earlier that I actually made the right call. Point #7 ("I find it confusing that other relevant decision-makers (eg, CDC, local school districts) are/were not more faster-moving") was a more significant source of concern earlier, but now it just seems like the public health authorities are a few days behind.
San Francisco Department of Public Health is not generally recommending the rescheduling of public events in San Francisco. However, those that involve vulnerable populations should be carefully reconsidered at this time. We advise all event planners to review CDC recommendations on mass gatherings released on March 2, 2020. If you are sick, you should stay home and not attend.
I'm personally less optimistic about advocacy especially in targeted scientific and medical domains, because I think there's a strong worry that the professionals should do their jobs (especially since they're likely to be overtaxed very soon) and advocates can easily be more harmful than helpful.
I'd feel more optimistic about this if EAs in biosecurity or public health strongly encourage this (though I understand if this isn't politically feasible for them to do so).
I'm personally less optimistic about advocacy, because I think there's a strong worry that the professionals should do their jobs (especially since they're likely to be overtaxed very soon) and advocates can easily be more harmful than helpful.
In addition to critical medication, I think it's plausibly a good idea to stock up on significantly more than one month of handwashing soap (and maybe other soap too). (Tentatively suggesting 1-2 years).
1. I'm not particularly worried about large-scale cuts for goods that have widespread availability (eg, food) and more worried about supply chain cuts for things that are relatively specific to coronavirus and other disaster-like scenarios.
2. (In America at least) Facemasks have been sold out for consumer use for several weeks; hand sanitizer has been somewhere between sold out and 10x more expensive in the last week. So some evidence that this will extend to other products.
3. Usage patterns are likely to change. For example I wouldn't be surprised if people (including EAs) use 4-6x less hand soap than they should.
4. If you anticipate increasing usage rates in the coming months, it's more prosocial to buy them now rather than in April/May so markets, factories, etc., have a chance to respond. (I find stockpiling masks much more iffy because 1. I think it's plausible that it'd be hard to repurpose other factories for facemasks, whereas for hand sanitizer and soap it's not hard to imagine relatively fast production changes, and 2. Outside of Asia, normal consumer usage of masks is pretty low so a sudden spike in demand might be harder to prepare for so it's better to leave them for medical service provdiers).
5. Selfishly, the cost of doing this is very low (you can always spend down your soap if this turns out to be a nothing-burger), while the potential benefits are large.
tl;dr: Ben estimates a personal counterfactual impact of 10 counterfactual votes in swing states for every hour of his labor (mostly software engineering), significantly higher than other plausible alternatives for work in politics.
Note that there was some pushback in the comments and also I think there's likely substantial diminishing marginal returns.
I'm coming into the conclusion that either finger guns, bows, or some other non-contact solution is better, or if we should give up on discouraging handshakes and instead encourage hand sanitizer immediately after washing hands.
I'm excited to see local groups experiment with trialing different practices and seeing which ones stick.
Handshakes are plausibly one of the more likely ways that infectious diseases spread, so it's worthwhile to look for healthier alternatives.
Over a year ago, someone asked the EA community whether it’s valuable to become world-class at an unspecified non-EA niche or field. Our Forum’s own Aaron Gertler responded in a post, saying basically that there’s a bunch of intangible advantages for our community to have many world-class people, even if it’s in fields/niches that are extremely unlikely to be directly EA-relevant.
Since then, Aaron became (entirely in his spare time, while working 1.5 jobs) a world-class Magic the Gathering player, recently winning the DreamHack MtGA tournament and getting $30,000 in prize monies, half of which he donated to Givewell.
I didn’t find his arguments overwhelmingly persuasive at the time, and I still don’t. But it’s exciting to see other EAs come up with unusual theories of change, actually executing on them, and then being wildly successful.
Publication bias alert: Not everybody liked the conference as much as I did. Someone I knew and respect thought some of the talks weren't very good (I agreed with them about the specific examples, but didn't think it mattered because really good ideas/conversations/networking at an event + gestalt feel is much more important for whether an event is worthwhile to me than a few duds).
That said, on a meta level, you might expect that people who really liked (or hated, I suppose) a conference/event/book to write detailed notes about it than people who were lukewarm about it.
Catalyst (biosecurity conference funded by the Long-Term Future Fund) was incredibly educational and fun.
Random scattered takeaways:
1. I knew going in that everybody there will be much more knowledgeable about bio than I was. I was right. (Maybe more than half the people there had PhDs?)
2. Nonetheless, I felt like most conversations were very approachable and informative for me, from Chris Bakerlee explaining the very basics of genetics to me, to asking Anders Sandberg about some research he did that was relevant to my interests, to Tara Kirk Sell detailing recent advances in technological solutions in biosecurity, to random workshops where novel ideas were proposed...
3. There's a strong sense of energy and excitement from everybody at the conference, much more than other conferences I've been in (including EA Global).
4. From casual conversations in EA-land, I get the general sense that work in biosecurity was fraught with landmines and information hazards, so it was oddly refreshing to hear so many people talk openly about exciting new possibilities to de-risk biological threats and promote a healthier future, while still being fully cognizant of the scary challenges ahead. I guess I didn't imagine there were so many interesting and "safe" topics in biosecurity!
5. I got a lot more personally worried about coronavirus than I was before the conference, to the point where I think it makes sense to start making some initial preparations and anticipate lifestyle changes.
6. There was a lot more DIY/Community Bio representation at the conference than I would have expected. I suspect this had to do with the organizers' backgrounds; I imagine that if most other people were to organize biosecurity conferences, it'd be skewed academic a lot more.
7. I didn't meet many (any?) people with a public health or epidemiology background.
8. The Stanford representation was really high, including many people who have never been to the local Stanford EA club.
9. A reasonable number of people at the conference were a) reasonably interested in effective altruism b) live in the general SF area and c) excited to meet/network with EAs in the area. This made me slightly more optimistic (from a high prior) about the value of doing good community building work in EA SF.
10. Man, the organizers of Catalyst are really competent. I'm jealous.
11. I gave significant amounts of money to the Long-Term Future Fund (which funded Catalyst), so I'm glad Catalyst turned out well. It's really hard to forecast the counterfactual success of long-reach plans like this one, but naively it looks like this seems like the right approach to help build out the pipeline for biosecurity.
12. Wow, evolution is really cool.
13. Talking to Anders Sandberg made me slightly more optimistic about the value of a few weird ideas in philosophy I had recently, and that maybe I can make progress on them (since they seem unusually neglected).
14. Catalyst had this cool thing where they had public "long conversations" where instead of a panel discussion, they'd have two people on stage at a time, and after a few minutes one of the two people get rotated out. I'm personally not totally sold on the format but I'd be excited to see more experiments like that.
15. Usually, conferences or other conversational groups I'm in have one of two failure modes: 1) there's an obvious hierarchy (based on credentials, social signaling, or just that a few people have way more domain knowledge than others) or 2) people are overly egalitarian and let useless digressions/opinions clog up the conversational space. Surprisingly neither happened much here, despite an incredibly heterogeneous group (from college sophomores to lead PIs of academic biology labs to biotech CEOs to DiY enthusiasts to health security experts to randos like me)
16. Man, it seems really good to have more conferences like this, where there's a shared interest but everybody come from different fields so it's less obviously hierarchal/status-jockeying.
17. I should probably attend more conferences/network more in general.
18. Being the "dumbest person in the room" gave me a lot more affordance to ask silly questions and understand new stuff from experts. I actually don't think I was that annoying, surprisingly enough (people seemed happy enough to chat with me).
19. Partially because of the energy in the conference, the few times where I had to present EA, I mostly focused on the "hinge of history/weird futuristic ideas are important and we're a group of people who take ideas seriously and try our best despite a lot of confusion" angle of EA, rather than the "serious people who do the important, neglected and obviously good things" angle that I usually go for. I think it went well with my audience today, though I still don't have a solid policy of navigating this in general.
20. Man, I need something more impressive on my bio than "unusually good at memes."
To be concrete, here are two problems I don't think the Hub's collection of resources currently fulfill.
1. My impression from looking through the content list on the EA Hub is that none of the sheets from the other groups can be directly adapted (even with significant modifications) for South Bay EA's audience, since the questions are either a) too broad and intro-level (like the CEA sheets) or b) have a lot of mandatory reading that's arguably not realistic for a heterogeneous group with many working professionals (eg the Harvard Arete stuff). I think SB EA is open to trying for more mandatory reading/high-engagement stuff among a subset of the members however. But right now if we are interested in an intermediate-level discussion on a topic that we haven't previously discussed (eg, geo-engineering, hinge of history), we basically have to make the sheets ourselves.
Historically we've found this to be true even for common topics that the online EA community has discussed for many years.
This isn't just a problem with the Hub to be clear; my group has been looking for a way to steal sheets from other groups since at least mid-2018. (It's possible our needs are really idiosyncratic but it'd be a bit of a surprise if that's true?)
2. I don't think of any of the existing sheets or guiding material as curriculum, per se. At least when we were creating the sheets, my co-organizers and I mostly did things that "seemed reasonable" through a combination of intuition and rough guesses/surveys about what our members liked. At no point did we have a strong educational theory or built things with an eye towards the latest advances in the educational literature. I suspect other local groups are similar to us in that when they created sheets and organized discussions, they tried their best with limited time and care, rather than have a strong theory of education or change.
If I were to design things from scratch, I'd probably want to work in collaboration with eg, educational or edutech professionals who are also very familiar with EA (some of whom have expressed interest in this). It's possible that EA material is so out-of-distribution that being familiar with the pedagogical literature isn't helpful, but I feel like it's at least worth trying?
For onlookers, I want to say I really appreciate bruno's top-level comment and that I have a lot of respect for bruno's contributions, both here and elsewhere. The comment I made two levels up was probably stronger than warranted and I really appreciate bruno taking it in stride, etc.
On a meta-level, in general I think your conversation with lucy is overly acrimonious, and it would be helpful to identify clear cruxes, have more of a scout's mindset, etc.
My read of the situation is that you (and other EAs upvoting or downvoting content) have better global priors, but lucy has more domain knowledge in the specific areas they chose to talk about.
I do understand that it's very frustrating for you to be in a developing country and constantly see people vote against their economic best interests, so I understand a need to vent, especially in a "safe space" of a pro-growth forum like this one.
However, lucy likely also feels frustrated about saying what they believe to be true things (or at least well-established beliefs in the field) and getting what they may perceive to be unjustifiably attacked by people who have different politics or epistemic worldviews.
My personal suggestion is to have a stronger "collaborative truth-seeking attitude" and engage more respectfully, though I understand if either you or lucy aren't up for it, and would rather tap out.