Posts

How can good generalist judgment be differentiated from skill at forecasting? 2020-08-21T23:13:12.132Z · score: 20 (6 votes)
What are some low-information priors that you find practically useful for thinking about the world? 2020-08-07T04:38:07.384Z · score: 56 (24 votes)
David Manheim: A Personal (Interim) COVID-19 Postmortem 2020-07-01T06:05:59.945Z · score: 31 (13 votes)
I'm Linch Zhang, an amateur COVID-19 forecaster and generalist EA. AMA 2020-06-30T19:35:13.376Z · score: 80 (39 votes)
Are there historical examples of excess panic during pandemics killing a lot of people? 2020-05-27T17:00:29.943Z · score: 28 (14 votes)
[Open Thread] What virtual events are you hosting that you'd like to open to the EA Forum-reading public? 2020-04-07T01:49:05.770Z · score: 16 (7 votes)
Should recent events make us more or less concerned about biorisk? 2020-03-19T00:00:57.476Z · score: 22 (9 votes)
Are there any public health funding opportunities with COVID-19 that are plausibly competitive with Givewell top charities per dollar? 2020-03-12T21:19:19.565Z · score: 25 (13 votes)
All Bay Area EA events will be postponed until further notice 2020-03-06T03:19:24.587Z · score: 25 (12 votes)
Are there good EA projects for helping with COVID-19? 2020-03-03T23:55:59.259Z · score: 31 (17 votes)
How can EA local groups reduce likelihood of our members getting COVID-19 or other infectious diseases? 2020-02-26T16:16:49.234Z · score: 23 (15 votes)
What types of content creation would be useful for local/university groups, if anything? 2020-02-15T21:52:00.803Z · score: 6 (1 votes)
How much will local/university groups benefit from targeted EA content creation? 2020-02-15T21:46:49.090Z · score: 24 (11 votes)
Should EAs be more welcoming to thoughtful and aligned Republicans? 2020-01-20T02:28:12.943Z · score: 31 (15 votes)
Is learning about EA concepts in detail useful to the typical EA? 2020-01-16T07:37:30.348Z · score: 42 (22 votes)
8 things I believe about climate change 2019-12-28T03:02:33.035Z · score: 59 (37 votes)
Is there a clear writeup summarizing the arguments for why deep ecology is wrong? 2019-10-25T07:53:27.802Z · score: 11 (6 votes)
Linch's Shortform 2019-09-19T00:28:40.280Z · score: 8 (2 votes)
The Possibility of an Ongoing Moral Catastrophe (Summary) 2019-08-02T21:55:57.827Z · score: 44 (23 votes)
Outcome of GWWC Outreach Experiment 2017-02-09T02:44:42.224Z · score: 14 (16 votes)
Proposal for an Pre-registered Experiment in EA Outreach 2017-01-08T10:19:09.644Z · score: 11 (11 votes)
Tentative Summary of the Giving What We Can Pledge Event 2015/2016 2016-01-19T00:50:58.305Z · score: 7 (7 votes)
The Bystander 2016-01-10T20:16:47.673Z · score: 5 (5 votes)

Comments

Comment by linch on Some thoughts on the effectiveness of the Fraunhofer Society · 2020-10-01T15:47:03.420Z · score: 6 (4 votes) · EA · GW

Great post! Do either you or other commentators here have a sense of how Fraunhofer compares to publicly funded research groups in other countries, like NASA, NIH, etc?

Also, have there been unusually strong success stories from other groups in that reference class?

Comment by linch on Linch's Shortform · 2020-09-30T00:55:48.428Z · score: 2 (1 votes) · EA · GW

Thanks for the encouragement and suggestion! Do you have recommendations for a really good title?

Comment by linch on [Linkpost] Some Thoughts on Effective Altruism · 2020-09-29T11:52:07.002Z · score: 3 (2 votes) · EA · GW

I think there aren't many joint root causes since so many of them are less about facts of the world and depend implicitly on your normative ethics. (As a trivial example, there's a sense in which the root cause of poverty, climate change and species extinctions is human population if you have an average utilitarian stance, but for many other aggregative views, trying to fix this will be abhorrent).

Some that I can think of:

1. A world primarily ruled by humans, instead of (as you say) "more ethical, reflective and/or rational" beings.

1a. evolution

1b. humans evolving from small-group omnivores instead of large-group herbivores

2. Coordination problems

3. Insufficient material resources

4. Something else?

I also disagree with the idea that "capitalism"(just to pick one example) is the joint root cause for most of the world's ills.

A. This is obviously wrong compared to something like evolution.

B. Global poverty predates capitalism and so does wild animal suffering, pandemic risk, asteroid risk, etc. (Also other problems commonly talked about like racism, sexism, biodiversity loss)

C. No obvious reason why non-capitalist individual states (in an anarchic world order) would not still have major coordination problems around man-made existential risks and other issues.

D. Indeed, we have empirical experience of the bickering and rising tensions between Communist states in the mid-late 1900s.

Comment by linch on [Linkpost] Some Thoughts on Effective Altruism · 2020-09-29T11:46:29.974Z · score: 5 (3 votes) · EA · GW

I really like the conception of thinking of root causes in terms of a "joint causal diagram!" Though I'd like to understand if this is an operationalization that leftist scholars would also agree with, at the risk of this being a "steelman" that is very far away from the intended purpose.

Still it's interesting to think about.

Comment by linch on Open and Welcome Thread: September 2020 · 2020-09-29T11:38:52.909Z · score: 11 (3 votes) · EA · GW

Welcome to the forum!

Comment by linch on Linch's Shortform · 2020-09-29T01:59:06.300Z · score: 17 (7 votes) · EA · GW

Do people have advice on how to be more emotionally resilient in the face of disaster?

I spent some time this year thinking about things that are likely to be personally bad in the near-future (most salient to me right now is the possibility of a contested election + riots, but this is also applicable to the ongoing Bay Area fires/smoke and to a lesser extent the ongoing pandemic right now, as well as future events like climate disasters and wars). My guess is that, after a modicum of precaution, the direct objective risk isn't very high, but it'll *feel* like a really big deal all the time.

In other words, being perfectly honest about my own personality/emotional capacity, there's a high chance that if the street outside my house is rioting, I just won't be productive at all (even if I did the calculations and the objective risk is relatively low).

So I'm interested in anticipating this phenomenon and building emotional resilience ahead of time so such issues won't affect me as much.

I'm most interested in advice for building emotional resilience for disaster/macro-level setbacks. I think it'd also be useful to build resilience for more personal setbacks (eg career/relationship/impact), but I naively suspect that this is less tractable.

Thoughts?

Comment by linch on The Intellectual and Moral Decline in Academic Research · 2020-09-29T00:58:54.576Z · score: 4 (2 votes) · EA · GW

willbradshaw made the exact same point, earlier, and had lower karma. What's up with that?

EDIT: Retracted because the parent comment is substantive in different ways. Still, acknowledging the earlier comment would've been nice!

Comment by linch on The Intellectual and Moral Decline in Academic Research · 2020-09-29T00:56:04.282Z · score: 3 (2 votes) · EA · GW

I agree that for topics where there are transparent, obvious, and one-sided financial incentives on one side, and the other side has approximate consensus among experts, I agree that the side with bad incentives (and are the numerical minority) are more suspicious.

However, when I think of industry funding for research, I mostly don't think of

a few dissenting voices are funded by industry to argue that point of view

I think more of stuff in the vein of people actually trying to figure things out (eg, BigTech funds a lot of applied and even theoretical CS research, especially in AI and distributed systems).

I'd just point to how industry dealt with things like asbestos, lead, tobacco, and greenhouse gas emissions

I don't know much about the other examples. I agree with greenhouse gases. My impression is that there was a lot of misinformation/bad science about vaping, and this was at least as much (and likely more) the fault of academic researchers as it was the fault of entrenched corporate interests.

Comment by linch on [Linkpost] Some Thoughts on Effective Altruism · 2020-09-26T09:56:27.512Z · score: 11 (8 votes) · EA · GW

I think I find myself confused about it means for something to have a "single root cause."Having not thought about it too much, I personally currently think the idea looks conceptually confused. I am not a philosopher; however here are some issues I have with this conception:

1. Definitional boundaries

First of all, I think this notion of causation is kinda confused in some important ways, and it'd be surprising to have discrete cleanly-defined causes to map well to a "single root cause" in a way/idea that is easy for humans to understand.

2. Most things have multiple "root causes"

Secondly, in practice I feel like mostly things I care about are due to multiple causes, at least if 1) you only use "causes" as defined in a way that's easy for humans to understand and 2) you only go back far enough to causal chains that are possible to act on. For example, there's a sense of the root cause of factory farming obviously being the Big Bang, but in terms of things we can act on, factory farming is caused by:

1) A particular species of ape evolved to have a preference for the flesh of other animals.

2) That particular species of ape have a great deal of control over other animals, and the external environment

3) Factory farming is currently the technology that can produce meat most efficiently and cost-effectively.

4) Producers of meat (mostly) just care about production efficiency and cost-effectiveness, not animal suffering.

5) The political processes and coordination mechanisms across species is primarily through parasitism and predation rather than more cooperative mechanisms.

6) The political processes and coordination mechanisms within a particular species of ape is such that it is permissible for producers of meat to cause animal suffering.

... (presumably many others that I'm not creative/diligent enough to list).

How do we determine which of the 6+ causes are "real" root causes?

From what I understand of the general SJ/activism milieu, the perception is that interventions that attempt to change #6 counts as "systemic change," but interventions that change #1, #2 (certain forms of AI alignment), #3 (plant-based/clean meat), #4 (moral circle expansion, corporate campaigns), #5 (uplifting, Hedonistic Imperative) do not. This seems awfully suspicious to me, as if people had a predetermined conclusion.

3. Monocausal diagrams can still have intervention points to act on, and it's surprising if the best intervention point/cause is the first ("root") one.

Thirdly, even if you (controversially, in my view) can draw a clean causal diagram such that a bad action is monocausal and there's a clean chain from A->B->C->...->{ bad thing}, in practice it is still not obvious to me (and indeed would be rather surprising!) if there's a definitive status of A as the "real" root cause, in a way that's both well-defined and makes it such that A is uniquely the only thing you can act on.

Comment by linch on The Intellectual and Moral Decline in Academic Research · 2020-09-25T20:18:49.992Z · score: 6 (4 votes) · EA · GW
So while I am concerned about inefficiencies in academic work and the waste of taxpayer dollars, I'm much more worried about the effects of corporate money on research.

Are there studies on whether corporate-funded research are of typically lower (or higher) quality than publicly-funded academic research? I can imagine it going either way, but I feel like I only have loose intuitions and anecdotes to go off of, and it'd be good to actually see data.

Comment by linch on [Open Thread] What virtual events are you hosting that you'd like to open to the EA Forum-reading public? · 2020-09-25T03:24:46.998Z · score: 2 (1 votes) · EA · GW

cross-posted from Facebook, which I will likely be checking much more regularly.

Forecasting 102 (EA SF discussion event)

Our previous forecasting workshop in April was a smashing success! Many people* have said it was helpful, insightful, fun etc. Can we repeat our success by having another great forecasting workshop? Only time (and a sufficiently large sample size) can tell!

Next Wednesday, EA SF is collaborating with Stanford Effective Altruism to host another event on forecasting. Together, we will practice forecasting techniques: the skills and ways of thinking that allow us to quantify our uncertainties and be slightly less confused about the future.

Some Background

https://en.wikipedia.org/wiki/The_Good_Judgment_Project

https://en.wikipedia.org/wiki/Superforecasting

http://www.academia.edu/download/37711009/2015_-_superforecasters.pdf

https://aiimpacts.org/evidence-on-good-forecasting-practices-from-the-good-judgment-project-an-accompanying-blog-post/

Here are some of the tentative topics we'll try to cover and practice in small groups, time permitting:

Question selection: How do we know what questions are the right ones to ask?

Question operationalization: How do we ask the right questions in the right way?

Intuition pumps & elicitation: How do we understand our intuitions in a way that’s accessible to our conscious thinking?

Information sources & internet research: How do we efficiently gather information in a time-constrained manner?

Distributional forecasting: How do we give probabilities on a range of outcomes, not just a single number for a distribution?

Technical tools: What tools are useful in aiding our forecasting?

As well as some general practice on calibration and making quantified predictions of the future!

Structure: We'll meet together briefly to go over the details and then split into smaller groups with 3-6 group members each, including a group leader. Each group will be given a discussion sheet that they can copy and group leaders will be given an answer key.

We'll be using Zoom as our platform as that allows the most seamless split into smaller groups. For people with security concerns, we recommend using the Zoom Web portal over the Mac/Windows App (I am uncertain of the quality of the app on Linuxes).

The assumed background is people with some passing familiarity with forecasting (eg, have attended a prior forecasting workshop by EA San Francisco or others, have done some predictions on metaculus, or have otherwise read Superforecasting), with some members having significantly more experience. However everybody’s welcome! If you have no prior exposure to forecasting, I recommend reading the AI impacts summary of Superforecasting[1] and doing some Open Phil calibration exercises[2]. Depending on who shows up, it might also make sense to have a Q&A in addition to the small group discussions.

As this is a virtual event, all are welcome. However, we only have limited small group leader capacity, so in the (very fortunate!) world where many more people show up than we expect, groups may be asked to nominate their own group leaders instead of having an appointed one with prior experience managing EA discussions.

Hope to see you there!

*n>=1, source unverified

[1] https://aiimpacts.org/evidence-on-good-forecasting-practices-from-the-good-judgment-project-an-accompanying-blog-post/

[2] https://www.openphilanthropy.org/blog/new-web-app-calibration-training

Comment by linch on Forecasting Thread: Existential Risk · 2020-09-24T20:10:23.885Z · score: 8 (3 votes) · EA · GW

I think this is a good point. I think people probably underrate the costs of duplicate/redundant work. That said:

1) You can't see detailed predictions of other individual people on Metaculus, only the aggregated prediction by one of Metaculus's favored weightings.

2) The commenting system on Metaculus is more barebones than the EA Forum or LessWrong (eg you can't attach pictures, there's no downvote functionality).

3) The userbases are different.

Comment by linch on What are words, phrases, or topics that you think most EAs don't know about but should? · 2020-09-24T11:51:14.867Z · score: 4 (2 votes) · EA · GW

Wow that's an awfully specific way to fail a job interview! But I'm glad you've learned something from it, at least?

Comment by linch on Linch's Shortform · 2020-09-24T08:48:56.001Z · score: 42 (16 votes) · EA · GW

Here are some things I've learned from spending the better part of the last 6 months either forecasting or thinking about forecasting, with an eye towards beliefs that I expect to be fairly generalizable to other endeavors.

Note that I assume that anybody reading this already has familiarity with Phillip Tetlock's work on (super)forecasting, particularly Tetlock's 10 commandments for aspiring superforecasters.

1. Forming (good) outside views is often hard but not impossible. I think there is a common belief/framing in EA and rationalist circles that coming up with outside views is easy, and the real difficulty is a) originality in inside views, and also b) a debate of how much to trust outside views vs inside views.

I think this is directionally true (original thought is harder than synthesizing existing views) but it hides a lot of the details. It's often quite difficult to come up with and balance good outside views that are applicable to a situation. See Manheim and Muelhauser for some discussions of this.

2. For novel out-of-distribution situations, "normal" people often trust centralized data/ontologies more than is warranted. See here for a discussion. I believe something similar is true for trust of domain experts, though this is more debatable.

3. The EA community overrates the predictive validity and epistemic superiority of forecasters/forecasting.

(Note that I think this is an improvement over the status quo in the broader society, where by default approximately nobody trusts generalist forecasters at all)

I've had several conversations where EAs will ask me to make a prediction, I'll think about it a bit and say something like "I dunno, 10%?"and people will treat it like a fully informed prediction to make decisions about, rather than just another source of information among many.

I think this is clearly wrong. I think in any situation where you are a reasonable person and you spent 10x (sometimes 100x or more!) time thinking about a question then I have, you should just trust your own judgments much more than mine on the question.

To a first approximation, good forecasters have three things: 1) They're fairly smart. 2) They're willing to actually do the homework. 3) They have an intuitive sense of probability.

This is not nothing, but it's also pretty far from everything you want in a epistemic source.

4. The EA community overrates Superforecasters and Superforecasting techniques. I think the types of questions and responses Good Judgment .* is interested in is a particular way to look at the world. I don't think it is always applicable (easy EA-relevant example: your Brier score is basically the same if you give 0% for 1% probabilities, and vice versa), and it's bad epistemics to collapse all of the "figure out the future in a quantifiable manner" to a single paradigm.

Likewise, I don't think there's a clear dividing line between good forecasters and GJP-certified Superforecasters, so many of the issues I mentioned in #3 are just as applicable here.

I'm not sure how to collapse all the things I've learned on this topic in a few short paragraphs, but the tl;dr is that I trusted superforecasters much more than I trusted other EAs before I started forecasting stuff, and now I consider their opinions and forecasts "just" an important overall component to my thinking, rather than a clear epistemic superior to defer to.

5. Good intuitions are really important. I think there's a Straw Vulcan approach to rationality where people think "good" rationality is about suppressing your System 1 in favor of clear thinking and logical propositions from your system 2. I think there's plenty of evidence for this being wrong*. For example, the cognitive reflection test was originally supposed to be a test of how well people suppress their "intuitive" answers to instead think through the question and provide the right "unintuitive answers", however we've later learned (one fairly good psych study. May not replicate, seems to accord with my intuitions and recent experiences) that more "cognitively reflective" people also had more accurate initial answers when they didn't have the time to think through the question.

On a more practical level, I think a fair amount of good thinking is using your System 2 to train your intuitions, so you have better and better first impressions and taste for how to improve your understanding of the world in the future.

*I think my claim so far is fairly uncontroversial, for example I expect CFAR to agree with a lot of what I say.

6. Relatedly, most of my forecasting mistakes are due to emotional rather than technical reasons. Here's a Twitter thread from May exploring why; I think I mostly still stand by this.

Comment by linch on How Dependent is the Effective Altruism Movement on Dustin Moskovitz and Cari Tuna? · 2020-09-23T04:11:07.910Z · score: 17 (10 votes) · EA · GW
Even if there was no funding at all, we could still accomplish a bunch (by going to work in existing institutions like govt., academia, non-profits, or working as volunteers).

Also existing EAs are doing or planning to do direct work in large part because of the existence of Open Phil funding. I expect that if Good Ventures decided to part ways, a fair number of people will pivot to earning-to-give instead.

Comment by linch on microCOVID.org: A tool to estimate COVID risk from common activities · 2020-09-22T00:41:15.490Z · score: 2 (1 votes) · EA · GW
And even a lowish probability of a long chain means the bulk of the damages are on other people rather than your self

Sure, but how large? At an empirical IFR of 0.5%, and expected chain size of 5 (which I think is a bit of an overestimate for most of my friends in Berkeley), you get to 2% fatality rate in expectation (assuming personal risk negligible).

If you assume local IFRs of your child nodes are smaller than global IFR, you can easily cut this again by 2-5x.

This is all empirical questions, before double-counting concerns in moral aggregation.

Comment by linch on EA Relationship Status · 2020-09-21T11:39:59.526Z · score: 2 (1 votes) · EA · GW

I think this was poorly phrased on my part. I meant to say "it is not only the case." I will edit the parent comment.

Comment by linch on EA Relationship Status · 2020-09-20T03:28:51.743Z · score: 5 (3 votes) · EA · GW

Information source: Not sure if this is the right reference class, but it's interesting to note that the most famous historical utilitarians seemed to have married late.

1. Jeremy Bentham was never married, and AFAIK has never had a romantic relationship.

2. John Stuart Mill married Harriet Mill (also a prominent utilitarian) when he was 45. She was 44. If his autobiography is to be believed, she was his first and only serious romantic interest.

3. Henry Sidgwich married at 38. (though some biographers think he was privately gay).

4. Bertrand Russell seemed to be a bit of an outlier, marrying at 22, 49, 64 and 80.

5. Derek Parfit married at 67.

6. Peter Singer (also an outlier) married at 22.

The earlier examples are especially interesting, because I'd expect the average age of marriage to be much lower, historically. Of course, they might also generalize less well.

Comment by linch on EA Relationship Status · 2020-09-20T03:12:42.834Z · score: 9 (2 votes) · EA · GW

This is interesting. The numbers here are not surprising based on my independent observations, but the phenomenon is in some sense fairly surprising. Several other considerations:

1. Anecdotally, conditional upon marriage, the rate of divorce in my EA friends seem much higher than among my non-EA friends of similar ages. So it is only not the case that EAs are careful/slow to marry because they are less willing to make long-term commitments that they cannot always keep, or because they are more okay with pre-marital cohabitation.

Obviously in any given case this should not be a cause of blame (in all the situations I have sufficient detail about, it appears that divorce was the best option in each of those cases). However, collectively the pattern should require some explanation.

2. Along with some of the other commenters, I share the anecdotes that my EA friends are much less likely to be married than my non-EA friends, or other groups. To add to the list of anecdotes, among Googlers who a) I know from non-EA contexts, eg former coworkers, b) are older than me and c) I know well enough to be >80% of confident of their relationship statuses, I think > 50% of them are married. I think the numbers are closer to 25-30% for Googlers of a similar age range I know through EA (with some nuances, like I know one person who probably would have been married if not for polyamory), and similar (if not slightly lower) numbers for non-Googler EAs I know well.

3. My inside view is that if you don't update on the observed data and just consider which characteristics will make EAs more or less likely to be married, I think there are a bunch of factors that push EAs towards "more"as opposed to less. Possibly controversial, but consider:

A. EAs are, on average, disproportionately high in traits that are seen as positive for long-term relationships/marriages in the broader population. This includes obvious traits like elite college attendance (speaking as someone who has not attended one), high earning potential, and intellectual engagement, but also subtler traits like having good relationships with their parents (which should be an indicator for being on average better at long-term relationships), general willingness to make sacrifices, communication ability, and willingness to try different things for conflict resolution.

B. You might expect this to be a signaling problem (maybe EAs have positive traits that are hard for others to discover), but I think the meta-level evidence is against this? For example, elite college backgrounds and intellectual ability are relatively transparent. You might also expect EAs to on average be healthier and more conventionally attractive than baseline (For example, Americans aged 20-39 are ~40% likely to be obese for both men and women, and I think the numbers are much lower in EA).

C. EAs are much more likely to be in international relationships than baseline, and the relative legal benefits of marriage are usually higher for international marriages than domestic marriages.

Comment by linch on EA Relationship Status · 2020-09-19T22:53:38.996Z · score: 5 (3 votes) · EA · GW

I moderately think this is the wrong approach on the meta-level.

1. We observe a phenomenon where X demographic is less likely to exhibit Y characteristic.

2. You're coming up with a list of explanations (E1, E2, E3) to explain why X is less likely to have Y, and then stopping when the variance is sufficiently explained.

3. However this ignores that there might be reasons for why your prior should be does X is more likely to have Y.

And on the object level, I agree with the other commentators that EAs often draw from groups that are less, rather than more, likely to be single.

Comment by linch on Long-Term Future Fund: September 2020 grants · 2020-09-19T00:36:53.408Z · score: 11 (4 votes) · EA · GW

I think it's possible that last year was just unusually slow for people (possibly pandemic-related?)

I looked at 3B1B (the only Youtube explainer series I'm familiar with) and since 2015 Grant has produced ~100 high quality videos, which is closer to ~20 videos/year than ~10/year.

I'm not familiar with the others.

and could plausibly be ~20% more productive in a year in terms of the main, highly-produced videos

I feel like this is low-balling potential year-to-year variation in productivity. My inside view is that 50-100% increases in productivity is plausible.

Comment by linch on Long-Term Future Fund: September 2020 grants · 2020-09-19T00:10:19.923Z · score: 4 (3 votes) · EA · GW

To be clear, I think your overall comment added to the discussion more than it detracts, and I really appreciate you making it. I definitely did not interpret your claims as an attack, nor did I think it's a particularly egregious example of a bravery framing. One reason I chose to comment here is because I interpreted (correctly, it appears!) you as someone who'd be receptive to such feedback, whereas if somebody started a bravery debate with a clearer "me against the immoral idiots in EA" framing I'd probably be much more inclined to just ignore and move on.

It's possible my bar for criticism is too low. In particular, I don't think I've fully modeled meta-level considerations like:

1) That by only choosing to criticize mild rather than egregious cases, I'm creating bad incentives.

2) You appear to be a new commenter, and by criticizing newcomers to the EA Forum I risk making the EA Forum less appealing.

3) That my comment may spawn a long discussion.

Nonetheless I think I mostly stand by my original comment.

Comment by linch on Long-Term Future Fund: September 2020 grants · 2020-09-18T23:53:57.589Z · score: 2 (1 votes) · EA · GW

Yeah that makes a lot of sense. I think the rest of your comment is fine without that initial disclaimer, especially with your caveat in the last sentence! :)

Comment by linch on Long-Term Future Fund: September 2020 grants · 2020-09-18T13:19:32.002Z · score: 7 (8 votes) · EA · GW

Meta: Small nitpick, but I would prefer if we reduce framings like

This is going to sound controversial here (people are probably going to dislike this but I'm genuinely raising this as a concern)

See Scott Alexander on Against Bravery Debates.

Comment by linch on Long-Term Future Fund: September 2020 grants · 2020-09-18T13:13:19.192Z · score: 6 (4 votes) · EA · GW

I also notice myself being confused about the output here. I suspect the difficulty of being good at Youtube outreach while fully understanding technical AI safety concepts is a higher bar than you're claiming, but I also intuitively would be surprised if it takes an average of 2+ months to produce a video (though perhaps he spends a lot of time on other activities?

This quote

for example, he’s already helping existing organizations produce videos about their ideas

alludes to this.

Comment by linch on Some thoughts on EA outreach to high schoolers · 2020-09-18T05:24:42.971Z · score: 12 (4 votes) · EA · GW

I think the set of values commonly ascribed to EA is both more totalizing and a stronger attractor state than most counterfactuals.

Comment by linch on microCOVID.org: A tool to estimate COVID risk from common activities · 2020-09-18T00:08:20.547Z · score: 2 (1 votes) · EA · GW

Right, I think the argument as written may not hold for the UK (and other locations with very low prevalence but R ~=1). My intuitions, especially in recent months, have mostly been formed from a US context (specifically California), where R has never been that far away from 1 (and current infectious prevalence closer to 0.5%).

That said, here are a bunch of reasons to argue against "Alice, an EA reading this forum post, being infected in London means Alice is responsible for 30 expected covid-19 infections (and corresponding deaths at 2020/08 levels)."

(For simplicity, this comment assumes an Rt ~= 1, a serial interval of ~one week, and a timeframe of consideration of 6 months)

1. Notably, an average Rt~=1 means that the median/mode is very likely 0. So there's a high chance that any given chain will either terminate before Alice infects anybody else, or soon afterwards. Of course, as EAs with aggregatively ethics, we probably care more about the expectation than the medians, so the case has to be made that we're less likely on average to infect others. Which brings us to...

2. Most EAs taking some precautions are going to be less likely to be infected than average, so their expected Rt is likely <1. See Owen's comment and responses. Concretely, if you have a 1% annualized covid budget for a year (10,000 microcovids), which I think is a bit on the high side for London, then you're roughly exposing yourself to 200 microcovids a week. At a baseline population prevalence of 500 microcovids, this means you have a ~40% chance of getting covid-19 in a week conditional upon your contacts having it, which (assuming a squared term) means P(Alice infects others | Alice is infected) is also ~40%.

Notably a lot of your risk comes from model uncertainty, as I mentioned in my comment to Owen, so the real expected Rt(Alice) > 0.4

As I write this out, under those circumstances I think a weekly budget of 200 microcovids a week is possibly too high for Alice.

However, given that I live in Berkeley, I strongly suspect that E(Number of additional people infected, other than Linch | Linch being infected) is < 1. (especially if you ignore housemates).

3. If your contacts are also cautious-ish people, many of them who are EAs and/or have read this post, they are likely to also take more precautions than average, so P(Alice's child nodes infecting others | Alice's child nodes being infected) is also lower than baseline.

4. There's also the classist aspect here, where most EAs work desk jobs and aren't obligated to expose themselves to lots of risks like being essential workers.

5. Morally, this will involve a bunch of double-counting. Eg, if you imagine a graph where Alice infects one person, her child node infects another person etc, for the next 6 months, you have to argue that Alice is responsible for 30 infections, her child node is responsible for 29, etc. Both fully counterfactual credit assignment and proposed alternatives have some problems in general, but in this covid-y specific case I don't think having an aggregate responsibility of 465 infections when only 30 people will be infected will make a lot of sense. (Sam made a similar point here, which I critiqued because I think there should be some time dependence, but I don't think time dependence should be total).

6. Empirical IFR rates have gone down, and are likely to continue doing so as a) medical treatment improves, b)people make mostly reasonable decisions with their lives (self-select on risk levels) plus c) reasonable probability of viral doses going down due to mask usages and the like.

7. As a related point to #3 and #6, I'd expect Alice's child nodes to be not just more cautious but also healthier than baseline (they are not randomly drawn from the broader population!).

8. There's suggestive evidence of substantial behavioral modulation (which is a large factor keeping Rt ~=1). If true, this means any marginal infection (or lack thereof) has less than expected effect as other people adjust behavior to take less or more risks.

___

Counterarguments, to argue that E(# of people infected| Alice is infected)>>30:

1. Maybe there's a nontrivial number of worlds where London infections spike again. In those worlds, assuming a stable Rt~=1 is undercounting. (and at 0.05% prevalence, a lot of E(#s infected) is dominated by the tails).

2. Maybe 6 months is too short of an expected bound for getting the pandemic under control in London (again tail heavy).

3. Reinfections might mess up these numbers.

___

A nitpick:

In London, 5-10% have been infected

Where are you getting this range? All the estimates I've seen for London are >10%, eg this home study and this convenience sample of blood donors.

Comment by linch on Pablo Stafforini’s Forecasting System · 2020-09-17T05:02:35.672Z · score: 3 (2 votes) · EA · GW
My expectation is that in the long run, this might be somewhat helpful, but the main reason I actually built this system is not so much to improve my forecasting performance but more to improve my forecasting efficiency. Instead of spending 2 or three hours per day haphazardly back and forth between questions, here I have a systematic, deliberate approach that I can follow every day and that allows me to accomplish at least as much, perhaps even more, in just a fraction of the time.

This seems like a really good system/way of thinking about things. I feel like I'm much more driven by interest/excitement than systematic, deliberate approaches, and my vague intuition (hah!) is that interest/excitement generally outperforms systematization/deliberation in the short run but vastly underperforms it in the long run.

Comment by linch on Pablo Stafforini’s Forecasting System · 2020-09-17T04:56:46.039Z · score: 9 (4 votes) · EA · GW
The most trivial one, and I guess the less glamorous one, is to defer to the community

Hmm one issue I have with deferring to the community is that even in situations where it's individually epistemically valid, it seems to me to be bad for group epistemics to not form your own position. An analogy I use is a stock market where everybody only invests in index funds.

Some Metaculus discussion here.

Comment by linch on Are social media algorithms an existential risk? · 2020-09-16T10:51:50.855Z · score: 2 (1 votes) · EA · GW
In particular, arguments of the form "I don't see how recommender systems can pose an existential threat" are at least as invalid as "I don't see how AGI can pose an existential threat"

Hold on for a second here. AGI is (by construction) capable of doing everything a recommender system can do plus presumably other things, so it cannot be the case that arguments for AGI posing an existential threat is necessarily weaker than recommender systems posing an existential threat.

Comment by linch on Buck's Shortform · 2020-09-13T18:03:46.582Z · score: 19 (8 votes) · EA · GW

1. At an object level, I don't think I've noticed the dynamic particularly strongly on the EA Forum (as opposed to eg. social media). I feel like people are generally pretty positive about each other/the EA project (and if anything are less negative than is perhaps warranted sometimes?). There are occasionally low-quality critical posts (that to some degree reads to me as status plays) that pop up, but they usually get downvoted fairly quickly.

2. At a meta level, I'm not sure how to get around the problem of having a low bar for criticism in general. I think as an individual it's fairly hard to get good feedback without also being accepting of bad feedback, and likely something similar is true of groups as well?

Comment by linch on Judgement as a key need in EA · 2020-09-13T11:55:31.661Z · score: 8 (4 votes) · EA · GW

People in this discussion may be interested in reading/contributing to my question on how to cleanly delineate the differences between good judgment and forecasting skill.

Comment by linch on How can good generalist judgment be differentiated from skill at forecasting? · 2020-09-13T11:54:17.576Z · score: 6 (3 votes) · EA · GW

To answer my own question, here is my best guess for how "good judgment" is different from "skill at forecasting."

Good judgment can roughly be divided within 2 mostly distinct clusters:

  • Forming sufficiently good world models given practical constraints.
  • Making good decisions on the basis of such (often limited) models.

Forecasting is only directly related to the former, and not the later (though presumably there are some general skills that are applicable to both). In addition, within the "forming good world models" angle, good forecasting is somewhat agnostic to important factors like:

  • Group epistemics. There are times where it's less important whether an individual has the right world models but that your group has access to the right plethora of models.
    • It may be the case that it's practically impossible for a single individual to hold all of them, so specialization is necessary.
  • Asking the right questions. Having the world's lowest Brier score on something useless is in some sense impressive, but it's not very impactful compared to being moderately accurate on more important questions.
  • Correct contrarianism. As a special case of the above two points, in both science and startups, it is often (relatively) more important to be right about things that others are wrong about than it is to be right about everything other people are right about.

___

Note that "better world models" vs "good decisions based on existing models" isn't the only possible ontology to break up "good judgment."

- Owen uses understanding of the world vs heuristics.
- In the past, I've used intelligence vs wisdom.

Comment by linch on Are there any other pro athlete aspiring EAs? · 2020-09-13T10:32:38.012Z · score: 6 (4 votes) · EA · GW

I think the optimization mentality is a really big deal. There's a reason the deliberate practice literature focused on the sports and arts. To the extent that this is translatable to other endeavors (as you and jsteinhardt alludes to), this can be a really big deal for optimization endeavors in EA.

I think this translates across codes

What does "code" mean in this context? Different language codes spoken among different sportspeople?

Ultimately the biggest snowball will be made with the buy-in of fans, but I also think this is a strength of the area - a lot of people seem to be strongly influenced by the opinions and actions of their sporting heroes. More influenced than makes sense, in my opinion, but this is a huge lever nonetheless

I think this makes a lot of sense. As Ryan and others have mentions, there might also be non-monetary EA goals that are useful as well, for example policy goals that are more cosmopolitan and future-oriented, or inspiring/mentoring future generations of researchers and policymakers.

Comment by linch on Challenges in evaluating forecaster performance · 2020-09-12T12:02:17.906Z · score: 3 (2 votes) · EA · GW
Suppose Alice encountered the important poll result because she was looking for it (as part of her effort to come up with a new forecast).

This makes Alice a better forecaster, at least if the primary metric is accuracy. (If the metric includes other factors like efficiency, then we need to know eg. how many more minutes, if any, Alice spends than Bob).

At the end of the day what we really care about is how much weight we should place on any given forecast made by Alice/Bob.

If Alice updates daily and Bob updates once a month, and Alice has a lower average daily Brier score, then all else being equal, if you saw their forecasts at a random day, you should trust Alice's forecasts more*.

If you happen to see their forecasts on the day Bob updates, I agree this is a harder comparison, but I also don't think this is an unusually common use case.

I think part of the thing driving our intuition differences here is that I think lack of concurrency of forecasts (timeliness of opinions) is often a serious problem "in real life," rather than just an artifact of the platforms. In other words, you are imagining that whether to trust Alice at time t vs Bob at time t-1 is an unfortunate side effect of forecasting platforms, and "in real life" you generally have access to concurrent predictions by Alice and Bob. Whereas I think the timeliness tradeoff is a serious problem in most attempts to get accurate answers.

If you're trying to decide whether eg, a novel disease is airborne, you might have the choice of a meta-analysis from several months back, an expert opinion from 2 weeks ago, a prediction market median that was closed last week, or a single forecaster's opinion today.

___

Griping aside, I agree that there are situations where you do want to know "conditional upon two people making a forecast at the same time, whose forecasts do I trust more?" There are different proposed and implemented approaches around this, for example prediction markets implicitly get around this problem since the only people trading are people who implicitly believe that their forecasts are current, so the latest trades reflect the most accurate market beliefs, etc. (though markets have other problems like greater fool, especially since the existing prediction markets are much smaller than other markets).

*I've noticed this in myself. I used to update my Metaculus forecasts several times a week, and climbed the leaderboard fairly quickly in March and April. I've since slowed down to averaging an update once 3-6 weeks for most questions (except for a few "hot" ones or ones I'm unusually interested in). My score has slipped as a result. On the one hand I think this is a bit unfair since I feel like there's an important "meta" sense in which I've gotten better (more intuitive sense of probability, more acquired subject matter knowledge on the questions I'm forecasting). On the other, I think there's a very real sense that alex alludes to in which LinchSeptember is just a worse object-level forecaster than LinchApril, even if in some important meta-level ones (I like to imagine) I've gotten better.

Comment by linch on Linch's Shortform · 2020-09-12T09:27:18.675Z · score: 13 (5 votes) · EA · GW

I'm worried about a potential future dynamic where an emphasis on forecasting/quantification in EA (especially if it has significant social or career implications) will have adverse effects on making people bias towards silence/vagueness in areas where they don't feel ready to commit to a probability forecast.

I think it's good that we appear to be moving in the direction of greater quantification and being accountable for probability estimates, but I think there's the very real risk that people see this and then become scared of committing their loose thoughts/intuitive probability estimates on record. This may result in us getting overall worse group epistemics because people hedge too much and are unwilling to commit to public probabilities.

See analogy to Jeff Kaufman's arguments on responsible transparency consumption:

https://www.jefftk.com/p/responsible-transparency-consumption

Comment by linch on Challenges in evaluating forecaster performance · 2020-09-10T00:34:18.026Z · score: 19 (5 votes) · EA · GW

Great post! As you allude to, I'm increasingly of the opinion that the best way to evaluate forecaster performance is via how much respect other forecasters give them. This has a number of problems:

  • The signal is not fully transparent: people who don't do at least a bit of forecasting (or are otherwise engaged with forecasters) will be at a loss about which forecasters others respect.
  • The signal is not fully precise: I can give you a list of forecasters I respect and a loose approximation of how much I respect them, but I'd be hard-pressed to give a precise rank ordering.
  • Forecasters are not immune to common failures of human cognition: we might expect demographic or ideological biases creep in on forecasters' evaluations of each other.
    • Though at least in the GJP/Metaculus style forecasting, a frequent pattern of (relative) anonymity hopefully alleviates this a lot
  • There are other systematic biases in subjective evaluation of ability that may diverge from "Platonic" forecasting skill
    • One that's especially salient to me is that (I suspect) verbal ability likely correlates much more poorly with accuracy than it does with respect.
    • I also think it's plausible that, especially in conversation, forecasters on average usually overweight complex explanations/nuance more than is warranted by the evidence.
  • It just pushes the evaluation problem up one level: how do forecasters evaluate each other?

However, as you mention, other metrics have as many if not more problems. So on balance, I think as of 2020, the metric "who do other forecasters respect" currently carries more signal than any other metric I'm aware of.

That said, part of me still holds out hope that "as of 2020" is doing most of the work here. Forecasting in many ways seems to me like a nascent and preparadigm field, and it would not shock me if in 5-15 years we have much better ontologies/tools of measurement so that (as with other more mature fields) more quantified metrics will be better in the domain of forecaster evaluation than loose subjective human impressions.

Alice, who starts off a poor forecaster but through diligent practice becomes a good (but not great) and regular contributor to a prediction platform has typically done something more valuable and praiseworthy than Bob

I think this is an underrated point. Debating praiseworthiness seems like it can get political real fast, but I want to emphasize the point about value: there are different reasons you may care about participation in a forecasting platform, for example:

  • "ranking" people on a leaderboard, so you can use good forecasters for other projects
  • you care about the results of the actual questions and the epistemic process used to gain those results.

For the latter use case, I think people who participate regularly on the forecasting platforms, contribute a lot of comments, etc, usually improve group epistemics much more than people who are unerringly accurate on just a few questions.

Metaculus, as you mention, is aware of this, and (relative to GJO) rewards activity more than accuracy. I think this has large costs (in particular I think it makes the leaderboard a worse signal for accuracy), but is still on balance better.

__

A side note about Goodhart's law: I directionally agree with you, but I think Goodhart's law (related: optimizer's curse, specification gaming) is a serious issue to be aware of, but (as with nuance) I worry that in EA discussions about Goodhart's law there's a risk of being "too clever." At any point you're trying to collapse the complex/subtle/multivariate/multidimensional nature of reality to a small set of easily measurable/quantifiable dimensions (sometimes just one), you end up losing information. You hope that none of the information you lose is particularly important, but in practice this is rarely true.

Nonetheless, it is the case that to (a first approximation), imperfect metrics often work in getting the things you want to get done. For example, the image/speech recognition benchmarks often have glaring robustness holes that are easy to point out, yet I think it's relatively uncontroversial that in many practical use cases, there are a plethora of situations where ML perception classifiers, created in large part by academics and industry optimizing along those metrics, are currently at or will soon approach superhuman quality.

Likewise, in many businesses, a common partial solution for principle-agent problems is for managers to give employees metrics of success (usually gameable ones that are only moderately correlated with the eventual goal of profit maximization). This can result in wasted effort via specification gaming, but nonetheless many businesses still end up being profitable as a direct result of employees having direct targets.

Perhaps best is a mix of practical wisdom and balance - taking the metrics as useful indicators but not as targets for monomaniacal focus. Some may be better at this than others.

I think (as with some of our other "disagreements") I am again violently agreeing with you. Your position seems to be "we should take metrics as useful indicators but we should be worried about taking them too seriously" whereas my position is closer to "we should be worried about taking metrics too seriously, but we should care a lot about the good metrics, and in the absence of good metrics, try really hard to find better ones."

Comment by linch on Open Philanthropy: Our Progress in 2019 and Plans for 2020 · 2020-09-06T11:14:30.172Z · score: 2 (1 votes) · EA · GW

Have you guys ended up doing this call? If so, do you feel like you have a (compressed) understanding and/or agreement with OpenPhil's position here?

Comment by linch on Suggest a question for Peter Singer · 2020-09-05T19:36:24.357Z · score: 9 (3 votes) · EA · GW

People in this discussion might be interested in my past interview with Singer.

Comment by linch on What are some low-information priors that you find practically useful for thinking about the world? · 2020-09-03T18:13:25.664Z · score: 3 (2 votes) · EA · GW

I'd be really excited if you were to do this.

Comment by linch on AMA: Owen Cotton-Barratt, RSP Director · 2020-09-03T00:17:23.974Z · score: 2 (1 votes) · EA · GW

Thanks a lot!

Comment by linch on AMA: Owen Cotton-Barratt, RSP Director · 2020-09-03T00:17:07.922Z · score: 5 (4 votes) · EA · GW

(I'm amused at the distribution of votes on this question).

Comment by linch on Some thoughts on the EA Munich // Robin Hanson incident · 2020-09-02T22:44:07.692Z · score: 2 (1 votes) · EA · GW

Hmm in my parent comment I said "structurally similar, though of course it is not exactly the same" which means I'm not defending that it's exactly a case. However upon a reread I actually think considering it a noncentral example is not too badly off. I think the following (the primary characterization of Copenhagen Interpretation of Ethics) is a fairly accurate representation:

when you observe or interact with a problem in any way, you can be blamed for it.

However it does not fill the secondary constraints Jai lays out:

At the very least, you are to blame for not doing more. Even if you don’t make the problem worse, even if you make it slightly better, the ethical burden of the problem falls on you as soon as you observe it. In particular, if you interact with a problem and benefit from it, you are a complete monster.

In this case, by choosing to invite a speaker and then (privately) cancelling it, they've indeed made the situation worse by a) wasting Hanson's time and b) mildly degraded professional norms.

But that level of badness seems on the whole pretty mediocre/mundane to first order.

Comment by linch on Some thoughts on the EA Munich // Robin Hanson incident · 2020-09-02T20:46:50.530Z · score: 6 (3 votes) · EA · GW

It didn't occur to me that the organization was CEA but I also didn't read it too carefully.

Comment by linch on What are some low-information priors that you find practically useful for thinking about the world? · 2020-09-02T12:44:04.932Z · score: 2 (1 votes) · EA · GW

Agreed, though I think most of the positive resolutions were closely related to covid-19?

Comment by linch on Some thoughts on the EA Munich // Robin Hanson incident · 2020-09-01T17:58:00.836Z · score: 2 (1 votes) · EA · GW

I am reasonably confident that this is the best first-order explanation.


EDIT: Habryka's comment makes me less sure that this is true.

Comment by linch on microCOVID.org: A tool to estimate COVID risk from common activities · 2020-09-01T01:58:52.516Z · score: 7 (4 votes) · EA · GW

What's the best way to give feedback? Your contact page said that tweeting is fine so I just left a small comment there.

I think doing chain analysis is hard because you basically need a full epi model, which isn't easy to do (especially in places where % infected is low) at an interesting granularity, since (from reading your white paper) your budget/target for model uncertainty seems to be <3x.

Comment by linch on microCOVID.org: A tool to estimate COVID risk from common activities · 2020-08-31T22:20:27.165Z · score: 2 (1 votes) · EA · GW

Glad to help! :)

Comment by linch on microCOVID.org: A tool to estimate COVID risk from common activities · 2020-08-31T20:52:21.836Z · score: 4 (2 votes) · EA · GW

I think this is definitely partially true.

That said, I have some discount factor in my intuitions but much less than a squared term. Part of the issue is that your (Bayesian) chances of infecting others is not independent of your chances of being infected, a fair fraction of my "probability of being infected" comes from model uncertainty so there's a substantial error term for correlated reasons to think that we're in some way wrong about how we are modeling risk.

Comment by linch on microCOVID.org: A tool to estimate COVID risk from common activities · 2020-08-31T20:43:40.786Z · score: 9 (2 votes) · EA · GW

This depends a lot on where and when you're situated!

For example, California's total infected numbers are plausibly >14% now, so reinfections aside, a chain of ~15 people from the marginal infection as of August 31 California is implausibly high.

Numbers aren't that different for London (as of late May), see page 22 of this report.

Likewise, the empirical fatality rate used to be >1% in the US in March/early April, but is likely lower than 0.5% in the US now, partially due to better treatment and mostly due to changing demographics in who gets infected (younger people less cautious and more likely to be infected, etc).

In contrast, I can totally believe that a marginal infection outside of East Asia/Oceana in mid-March will result in >20 infections.