RP Work Trial Output: How to Prioritize Anti-Aging Prioritization - A Light Investigation 2021-01-12T22:51:31.802Z
Some learnings I had from forecasting in 2020 2020-10-03T19:21:40.176Z
How can good generalist judgment be differentiated from skill at forecasting? 2020-08-21T23:13:12.132Z
What are some low-information priors that you find practically useful for thinking about the world? 2020-08-07T04:38:07.384Z
David Manheim: A Personal (Interim) COVID-19 Postmortem 2020-07-01T06:05:59.945Z
I'm Linch Zhang, an amateur COVID-19 forecaster and generalist EA. AMA 2020-06-30T19:35:13.376Z
Are there historical examples of excess panic during pandemics killing a lot of people? 2020-05-27T17:00:29.943Z
[Open Thread] What virtual events are you hosting that you'd like to open to the EA Forum-reading public? 2020-04-07T01:49:05.770Z
Should recent events make us more or less concerned about biorisk? 2020-03-19T00:00:57.476Z
Are there any public health funding opportunities with COVID-19 that are plausibly competitive with Givewell top charities per dollar? 2020-03-12T21:19:19.565Z
All Bay Area EA events will be postponed until further notice 2020-03-06T03:19:24.587Z
Are there good EA projects for helping with COVID-19? 2020-03-03T23:55:59.259Z
How can EA local groups reduce likelihood of our members getting COVID-19 or other infectious diseases? 2020-02-26T16:16:49.234Z
What types of content creation would be useful for local/university groups, if anything? 2020-02-15T21:52:00.803Z
How much will local/university groups benefit from targeted EA content creation? 2020-02-15T21:46:49.090Z
Should EAs be more welcoming to thoughtful and aligned Republicans? 2020-01-20T02:28:12.943Z
Is learning about EA concepts in detail useful to the typical EA? 2020-01-16T07:37:30.348Z
8 things I believe about climate change 2019-12-28T03:02:33.035Z
Is there a clear writeup summarizing the arguments for why deep ecology is wrong? 2019-10-25T07:53:27.802Z
Linch's Shortform 2019-09-19T00:28:40.280Z
The Possibility of an Ongoing Moral Catastrophe (Summary) 2019-08-02T21:55:57.827Z
Outcome of GWWC Outreach Experiment 2017-02-09T02:44:42.224Z
Proposal for an Pre-registered Experiment in EA Outreach 2017-01-08T10:19:09.644Z
Tentative Summary of the Giving What We Can Pledge Event 2015/2016 2016-01-19T00:50:58.305Z
The Bystander 2016-01-10T20:16:47.673Z


Comment by linch on The Upper Limit of Value · 2021-01-27T18:52:15.991Z · EA · GW

Thank you so much for this paper! I literally made a similar argument to someone last weekend (in the context of economic growth), glad to have a canonical/detailed source to look at so I can present more informed views/have an easy thing to link to.

I will read the rest of the paper later, but just flagging that I don't find your response to "incomplete understanding of physics" particularly persuasive:

Perhaps our understanding of physics is incorrect. That is, it is possible that our understanding of any of the assumed correct disciplines discussed here, from cosmology to computation. This is not merely an objection to the authors’ personal grasp of the subjects, but a claim that specific premises may, in the future, be found to be incorrect.

I think the strongest version of the "we don't understand physics" argument is that we (or at least I) have nonzero credence in physics as we know it to be mistaken in a way that allows for infinities. This results in an infinite expected value. 

Now, perhaps we can exclude arbitrarily exclude sufficiently small probabilities ("Pascal's mugging"). But at least for me, my inside-view credence in misunderstanding the finitude of physics is >0.1%, and I don't think Pascal's mugging exceptions should be applicable to probabilities at anywhere near that level.

Michael Dickens has a different issue where finite distributions can still have  infinite expected value, but I have not read enough of your paper to know if it addresses this objection.


Comment by linch on Long-Term Future Fund: Ask Us Anything! · 2021-01-26T03:17:05.346Z · EA · GW

(Not sure if this is the best place to ask this. I know the Q&A is over, but on balance I think it's better for EA discourse for me to ask this question publicly rather than privately, to see if others concur with this analysis, or if I'm trivially wrong for boring reasons and thus don't need a response). 

Open Phil's Grantmaking Approaches and Process has the 50/40/10 rule, where (in my medicore summarization), 50% of a grantmaker's grants have to have the core stakeholders (Holden Karnofsky from Open Phil and Cari Tuna from Good Ventures) on board, 40% have to be grants where Holden and Cari are not clearly on board, but can imagine being on board if they knew more, and  up to 10% can be more "discretionary." 
Reading between the lines, this suggests that up to 10% of funding from Open Phil will go to places Holden Karnofsky and Cari Tuna are not inside-view excited about, because they trust the grantmakers' judgements enough. 

Is there a similar (explicit or implicit) process at LTFF?

I ask because 

  • part of the original pitch for EA Funds, as I understood it, was that it would be able to evaluate higher-uncertainty, higher-reward donation opportunities that individual donors may not be equipped to evaluate.
  • Yet there's an obvious structural incentive to make "safer" and easier-to-justify-to-donors decisions.
  • When looking at the April, September, and November 2020 reports, none of the grants look obviously dumb, and there's only one donation that I feel moderately confident in disagreeing with.
  • Now perhaps both I and the LTFF grantmakers are unusually enlightened individuals, and accurately converged independently to great donation opportunities given the information available. Or I coincidentally share the same taste and interests. But it seems more likely that the LTFF is somewhat bounding the upside by making grants that seems good to informed donors on a first glance with public information in addition to grants that are good for very informed grantmakers upon careful reflection and private information. This seems suboptimal if true.
  • A piece of evidence for this view is that the April 2019 grants seems more inside-view intuitively suspicious to me at the time (and judging from the high density of critical comments on that post, this opinion is shared by many others on the EA Forum).
  • Now part of this is certainly that both LTFF and the EA community were trying to "find its feet" so to speak, and there was less of a shared social reality for what LTFF ought to do. And nowadays we're more familiar with funding independent researchers and projects like that.
  • However, I do not think this is the full story.
  • In general, I think I'm inclined to encourage the LTFF to become moderately more risk-seeking. In particular (if I recall my thoughts at the time correctly, and note that I have far from perfect memory or self-knowledge), I think if I were to rank the "most suspicious" LTFF grants in April 2019, I would have missed quite a few grants that I now think are good (moderate confidence). This suggests to me that moderately informed donors are not in a great spot to quickly evaluate the quality of LTFF grants.
Comment by linch on Evidence on correlation between making less than parents and welfare/happiness? · 2021-01-25T22:36:23.117Z · EA · GW

Hi Jessica, 

You might be interested in the latest study which seems to suggest that both life satisfaction and perceived well-being increases proportionally with log(income), though the former has larger correlation than the latter,  at least for employed Americans in this large-scale study: 

From the paper:

Data are from (18), a large-scale project using the experience sampling method (19, 20), in which participants’ smartphones were signaled at randomly timed moments during their waking hours and prompted to answer questions about their experience at the moment just before the signal. The present results are based on 1,725,994 reports of experienced well-being from 33,391 employed (emphasis mine), working-age adults (ages 18 to 65) living in the United States. Experienced well-being was measured with the question “How do you feel right now?” on a continuous response scale with endpoints labeled “Very bad” and “Very good,” while evaluative well-being was measured with the question, “Overall, how satisfied are you with your life?” on a continuous response scale with endpoints labeled “Not at all” and “Extremely.”


To formally assess whether experienced well-being plateaued around incomes of $75,000/y, the association between income and experienced well-being was analyzed separately for incomes below and above $80,000/y (the upper bound of the income band containing $75,000). Results showed that the slope of the association between log(income) and experienced well-being was virtually identical for incomes below and up to $80,000/y (b = 0.109, P < 0.00001) as it was for incomes larger than $80,000/y (b = 0.110, P < 0.00001). Although both forms of well-being rose linearly with log(income), the correlation was stronger for evaluative well-being (r = 0.17) than experienced well-being (r = 0.09, Pdifference < 0.00001).


Comment by linch on Chi's Shortform · 2021-01-22T23:24:18.459Z · EA · GW

If my comment didn't seem pertinent, I think I most likely misunderstood the original points then. Will reread and try to understand better.

Comment by linch on Chi's Shortform · 2021-01-22T22:04:52.349Z · EA · GW

Why is this comment downvoted? :)

Comment by linch on Chi's Shortform · 2021-01-20T23:56:30.095Z · EA · GW

I think within EA, people should report their accurate levels of confidence, which in some cultures and situations will come across as underconfident and in other cultures and situations will come across as overconfident. 

I'm not sure what the practical solution is to this level of precision bleeding outside of EA; I definitely felt like there were times where I was socially penalized for trying to be accurate in situations where accuracy was implicitly not called for. If I was smarter/more socially savvy the "obvious" right call would be to quickly codeswitch between different contexts, but in practice I've found it quite hard.


Separate from the semantics used, I agree there is a real issue where some people are systematically underconfident or overconfident relative to reality, and this hurts their ability to believe true things or achieve their goals  in the long run. Unfortunately this plausibly correlates with demographic differences (eg women on average less confident than men, Asians on average less confident than Caucasians), which seems worth correcting for if possible.

Comment by linch on What is going on in the world? · 2021-01-20T19:36:18.206Z · EA · GW

Thanks for your comment!

I agree. But you see, in some population dynamics, variation is correlated with increased risk of extinction.

I think I don't follow your point. If I understand correctly, the linked paper (at least from the abstract, I have not read it) talks about population-size variation, which has an intuitive/near-tautological relationship with increased risk of extinction, rather than variation overall. 

That might be precisely part of the problem. 

Sorry can you specify more what the problem is? If you mean that the problem is an inefficient distribution of limited resources, I agree that it's morally bad that I have access to a number of luxuries while others starve, and the former is casually upstream of the latter. However, in the long run we can only get maybe 1-2 orders of magnitude gains from a more equitable distribution of resources globally (though some rich  individuals/gov'ts can create more good than that by redistributing their own resources), but we can get much more through other ways to create more stuff/better experiences. 

We are just starting to be seriously concerned about the externalities of this increase in 


Who's this "we?" :P

Comment by linch on Training Bottlenecks in EA (professional skills) · 2021-01-20T08:30:30.465Z · EA · GW

One explanation for this is that organization-specific knowledge ... is valuable, but general-purpose skills aren't as valuable

I've also heard the explanation that firms are strongly incentivized to teach organization-specific knowledge but not general-purpose skills, because the former increases employee efficacy in the organization but the latter makes them more hirable at other organizations. 

This is obviously theoretically true but I haven't seen the literature on the effect size/am unclear how big an issue this is in practice.

Comment by linch on What is going on in the world? · 2021-01-19T08:04:38.760Z · EA · GW

I'm curious if there's a point about energy use that's large enough to be added to the list. Intuitively I think no (for the same reason that climate change doesn't seem as important as the above points), but on the scale of centuries, the story of humanity is intertwined with the story of energy use, so perhaps on an outside view this is just actually really underrated and important.

Comment by linch on What is going on in the world? · 2021-01-19T08:01:56.161Z · EA · GW

We (most humans in most of the world) lived or are living in a golden age, with more material prosperity and better physical health* than ever before. 2020 was shitty, and the second derivative might be negative, but the first derivative still looks clearly positive on the timescale of decades, as well as a (measured from history, not counterfactual) really high baseline. On a personal level, my consumption is maybe 2 orders of magnitude higher than that of my grandparents  at my age(might become closer to 3 if I was less EA). So I'd be interested in adding a few sentences like:

  • For the first time in recorded history, the vast majority of humans are much richer than their ancestors.
  • Even in the midst of a raging pandemic, human  deaths from infectious disease still account for less than 1/3 of all deaths.
  • People have access to more and better information than ever before.

I think as EAs, it's easy to have a pretty negative view of the world (because we want to fix on what we can fix, and also pay attention to a lot of things we currently can't fix in the hopes that one day we can figure out what to fix later), but obviously there is still a lot of good in the world (and there might be much more to come), and it might be valuable to have concrete reminders of what we ought to cherish and protect.

* I think it's plausible/likely that we're emotionally and intellectually healthier as well, but this case is more tenuous. 

Comment by linch on Clarifying the core of Effective Altruism · 2021-01-17T20:30:34.780Z · EA · GW

I think you're saying that my word choice is unusual here for commonsensical intuitions, but I don't think it is? Tennis is an unusually objective field, with clear metrics and a well-defined competitive system.

When somebody says "I think Barack Obama (or your preferred presidential candidate) is the best man to be president" I highly doubt that they literally mean there's a >50% chance that of all  living America-born citizens >35 years of age, this person will be better at governing the US than everybody else. 

Similarly, when somebody says "X is the best fiction author," I doubt they are expressing >50% credence that of all humans who have ever told a story, X told the best fiction stories. 

The reference class is the same as the field. Sorry I was clear. But like you said, there are >7 billion people, so "specific reference member" means something very different than "field overall."

Comment by linch on Clarifying the core of Effective Altruism · 2021-01-17T16:35:06.923Z · EA · GW

I guess when I think "best action to do" the normative part of the claim is something about the local map rather than the territory or the global map. I think this has two parts:

1) When I say "X is the best bet" I meant that my subjective probability P(X is best) > P(any specific other reference member). I'm not actually betting it against "the field" or claiming P(X is best) > 0.5!

2) If I believe that X is the best bet in the sense of highest probability, of course if I was smarter and/or had more information my assigned probabilities will likely change. 

Comment by linch on What actually is the argument for effective altruism? · 2021-01-16T21:04:32.872Z · EA · GW

On one hand, it might be the case that the actions which are most cost-effective at doing good actually do very little good, but are also very cheap (e.g. see this post by Hanson). Alternatively, maybe the most cost-effective actions are absurdly expensive, so that knowing what they are doesn't help us.

I think the first argument can be rescued by including search costs in the "cost" definition. I agree that the second one cannot be, and is a serious issue with this phrasing.

Comment by linch on Clarifying the core of Effective Altruism · 2021-01-16T21:01:36.467Z · EA · GW

Thanks for the post! I think disambiguating "EA as trajectory change" and "EA as hits-based giving" is particularly valuable for me.

In the face of radical uncertainty about the future, it seems hard to ever justifiably claim that one course of action is the “best thing to do”, rather than just a very good thing to do.

I'm confused by this. I assume that the "best thing to do" phrase is used ex ante rather than ex post.  Perhaps you're using the word "justifiably" to mean something more technical/philosophical than what the common language meaning is?

Comment by linch on The Folly of "EAs Should" · 2021-01-10T20:32:49.012Z · EA · GW

It has a negative effect on the local theater, but hopefully a positive effect on the counterfactual recipients of that money.

Comment by linch on The Folly of "EAs Should" · 2021-01-10T20:32:03.065Z · EA · GW

I'm generally leery of putting words in other people's mouths, but perhaps people are using "bad advice" to mean different things, or at least have different central examples in mind. 

There's at least 3 possible interpretations of what "bad advice" can mean here:

A. Advice that, if some fraction of people are compelled to follow it across the board, can predictably lead to worse outcomes than if the advice isn't followed.

B. Advice that, if followed by people likely to follow such advice, can predictably lead to worse outcomes than if the advice isn't followed.

C. Words that can be in some sense considered "advice" that have negative outcomes/emotional affect upon hearing these words, regardless of whether such advice is actually followed.

Consider the following pieces of "advice":

  1. You should self-treat covid-19 with homeopathy.
  2. You should eat raw lead nails.

#1 will be considered "bad advice" in all 3 interpretations (it will be bad if everybody treats covid-19 with homepathy(A), it will be bad if people especially susceptible to homeopathic messaging treat covid-19 with homeopathy(B), and also I will negatively judge someone for recommending self treatment with homeopathy(C)).

#2 is only "bad advice" in at most 2 of the interpretations (forcibly eating raw lead nails is bad(A), but realistically I don't expect anybody to listen to such "recommendations" ( B), and this advice is so obviously absurd that context will determine whether I'd be upset about this suggestion (C)). 

In context here, if Habryka (and for that matter me) doesn't know any EA ex-doctors who regret no longer being a doctor (whereas he has positive examples of EA ex-doctors who do not regret this), this is strong evidence that telling people to not be doctors is good advice under interpretation B*, and moderate-weak evidence that it's good advice under interpretation A.

(I was mostly reading "bad advice" in the context of B and maybe A when I first read these comments).

However, if David/Khorton interpret "bad advice" to mean something closer to C, then it makes more sense why not knowing a single person harmed by following such advice is not a lot of evidence for whether the advice is actually good or bad.

* I suppose you can posit a selection-effected world where there's a large "dark matter" of former EAs/former doctors who quit the medical profession, regretted that choice, and then quit EA in disgust. This claim is not insane to me, but will not be where I place the balance of my probabilities.

Comment by linch on Strong Longtermism, Irrefutability, and Moral Progress · 2021-01-08T11:52:08.865Z · EA · GW

My credence could be that working on AI safety will reduce existential risk by 5% and yours could be 10^-19%, and there’s no way to discriminate between them.

We can look at their track record on other questions, and see how reliably (or otherwise) different people's predictions track reality.

I agree that below a certain level (certainly by 10^-19, and possibly as high as 10^-3) direct calibration-in-practice becomes somewhat meaningless. But we should be pretty suspicious of people claiming extremely accurate probabilities at the 10^-10 mark if they aren't even accurate at the 10^-1 mark. 

In general I'm not a fan of this particular form of epistemic anarchy where people say that they can't know anything with enough precision under uncertainty to give numbers, and then act as if their verbal non-numeric intuitions are enough to carry them through consistently making accurate decisions. 

It's easy to lie (including to yourself) with numbers, but it's even easier to lie without them.

Comment by linch on My mistakes on the path to impact · 2021-01-07T10:01:47.479Z · EA · GW

Apologies for the long delay in response, feel free to not reply if you're busy.

Hmm I still think we have a substantive rather than framing disagreement (though I think it is likely that our disagreements aren't large). 

This is because very loosely speaking I expect not deferring to often be better if the stakes are concentrated on oneself and more deference to be better if one's own direct stake is small. I used a decision with large effects on others largely because then it's not plausible that you yourself are affected by a similar amount; but it would also apply to a decision with zero effect on yourself and a small effect on others. Conversely, it would not apply to a decision that is very important to yourself (e.g. something affecting your whole career trajectory).

Perhaps this heuristic is really useful for a lot of questions you're considering. I'm reminded of AGB's great quote:

There are enough individual and practical considerations here (in both directions) that in many situations the actual thing I would advocate for is something like “work out what you would do with both approaches, check against results ‘without fear or favour’, and move towards whatever method is working best for you”.

For me personally and the specific questions I've considered, I think considering whether/how much to defer to by dividing into buckets of "how much it affects myself or others" is certainly a pretty useful heuristic in the absence of better heuristics, but it's mostly superseded by a different decomposition:

  1. Epistemic -- In a context-sensitive manner, do we expect greater or lower deference in this particular situation to lead to more accurate beliefs.
  2. Role expectations* -- Whether the explicit and implicit social expectations on the role you're assuming privilege deference or independence. 

So I think a big/main reason it's bad to defer completely to others (say 80k) on your own career reasons is epistemic: you have so much thought and local knowledge about your own situation that your prior should very strongly be against others having better all-things-considered  views on your career choice than you do. I think this is more crux-y for me than how much your career trajectory affects yourself vs others (at any rate hopefully as EAs our career trajectories affect many others anyway!). 

On the other hand, I think my Cochrane review example above is a good epistemic example of deference. even though my dental hygiene practices mainly affect myself and not others (perhaps my past and future partners may disagree), I contend it's better to defer to the meta-analysis over my own independent analysis in this particular facet of my personal life.

The other main (non-epistemic) lens I'd use to privilege greater or lower humility is whether the explicit and implicit social expectations privilege deference or  independence. For example, we'd generally** prefer government bureaucrats in most situations to implement policies, rather than making unprincipled exceptions based on private judgements. This will often look superficially similar to "how much this affects myself or others." 

An example of a dissimilarity is when someone filling out a survey. This is a situation where approximately all of the costs and benefits are borne by other people. So if you have a minority opinion on a topic, it may seem like the epistemically humble-and-correct action is to fill out the poll according to what you believe the majority to think (or alternatively, fill it out with the answer that you privately think is on the margin more conducive to advancing your values). 

But in all likelihood, such a policy is one-thought-too-many, and in almost all situations it'd be more prudent to fill out public anonymous polls/surveys with what you actually believe.  

I agree that one should act such that one's all-things-considered view is that one is making the best decision (the way I understand that statement it's basically a tautology).

Agreed, though I mention this because in discussions of epistemic humility-in-practice, it's very easy to accidentally do double-counting.

*I don't like this phrase, happy to use a better one.

**I'm aware that there are exceptions, including during the ongoing coronavirus pandemic.

Comment by linch on How modest should you be? · 2021-01-06T18:45:25.109Z · EA · GW

Thanks for the compliment!

check against results ‘without fear or favour’, and move towards whatever method is working best for you.

Yeah that makes sense! I think this is a generally good approach to epistemics/life.

Comment by linch on How modest should you be? · 2021-01-05T01:05:03.971Z · EA · GW

Thanks for the reply! One thing you and AGB reminded me of that my original comment elided over is that some of these personal and "practical" considerations apply in both directions. For example for #4 there are many/most cases where understanding expert consensus is easier rather than harder than coming up with your own judgment.

It'd perhaps be interesting if  people produced a list of the most important/common practical considerations in either direction, though ofc much of that will be specific to the individual/subject matter/specific situation.

Comment by linch on Insomnia: a promising cure · 2021-01-04T21:23:43.308Z · EA · GW

Some people in the comments were recommending Why We Sleep.  People may be interested in this update by Alexey Guzey:

Matthew Walker's "Why We Sleep" Is Riddled with Scientific and Factual Errors

Here's an excerpt:

Any book of Why We Sleep’s length is bound to contain some factual errors. Therefore, to avoid potential concerns about cherry-picking the few inaccuracies scattered throughout, in this essay, I’m going to highlight the five most egregious scientific and factual errors Walker makes in Chapter 1 of the book. This chapter contains 10 pages and constitutes less than 4% of the book by the total word count.


No, two-thirds of adults in developed nations do not fail to obtain the recommended amount of sleep

Suppose that you recommend that adults sleep 7-9 hours per night.

  1. then, someone learns (a) that roughly 40% of people sleep less than 7 hours, roughly 25% sleep 7 hours, and roughly 35% sleep 8 hours or more, meaning that a bit over one-third of people sleep less than you recommend Linked data is for the US but it appears (a) that other developed countries have very similar sleep habits.
  2. then they look at your recommendation and say that you recommended an average of 8 hours of sleep per night.
  3. then they say that you recommended 8 hours of sleep per night
  4. then they say that two-thirds of people sleep less than the 8 hours you recommended

Would this be a fair representation of your position and of the data or would this be misleading?

This is literally what Walker does in his book. On page 3, in the very first paragraph of Chapter 1, Walker writes:

Two-thirds of adults throughout all developed nations fail to obtain the recommended eight hours of nightly sleep.

In the footnote to this sentence he writes:

The World Health Organization and the National Sleep Foundation both stipulate an average of eight hours of sleep per night for adults.

Here are the National Sleep Foundation’s sleep recommendations (a) announced in 2015:

Adults (26-64): Sleep range did not change and remains 7-9 hours

Here are the World Health Organization’s sleep recommendations:

The quote is empty because the WHO does not stipulate how much an adult should sleep anywhere. I don’t know where Walker got this information.

The rest of the post is written in a similarly exasperated and exhaustive nature, patiently taking down Walker's unfounded claims page by page. (Again, all of this is just in Chapter 1).

While I have not individually vetted the claims in Guzey's blog post, on face value it seems reasonably well-researched and careful, certainly more so than the book it was critiquing. There are also followups on the StatModelling blog by Andrew Gelman, a famous Bayesian statistician and blogger.

Note that Guzey's critique and the followups on Gelman's blog came out after this forum post and most of the associated comments were published, and it will be somewhat unfair to blame commentators for not being aware of the scientific and factual errors of an acclaimed/professed "sleep scientist."

Comment by linch on Forecasts about EA organisations which are currently on Metaculus. · 2021-01-02T12:28:32.388Z · EA · GW

Not quite at the point where I think additional work is helpful (eg the question operationalization stage), but want to flag that I and others at Rethink are actively excited about incorporating more forecasting in our workflow, and it's plausible that the format here (public Metaculus questions about organization metrics) would be relevant for us! 

Comment by linch on How modest should you be? · 2020-12-31T00:45:30.578Z · EA · GW

This isn't directly related to your point, but I think there are a number of practical issues with most attempts at epistemic modesty/deference, that theoretical approaches like this one do not adequately account for. 

1) Misunderstanding of what experts actually mean. It is often easier to defer to a stereotype in your head than to fully understand an expert's views, or a simple approximation thereof. 

Dan Luu gives the example of SV investors who "defer" to economists on the issue of discrimination in competitive markets without actually understanding (or perhaps reading) the relevant papers. 

In some of those cases, it's plausible that you'd do better trusting the evidence of your own eyes/intuition over your attempts to understand experts.

2) Misidentifying the right experts. In the US, it seems like the educated public roughly believes that "anybody with a medical doctorate" is approximately the relevant expert class on questions as diverse as nutrition, the fluid dynamics of indoor air flow (if the airflow happens to carry viruses), and the optimal allocation of limited (medical)  resources. 

More generally, people often default to the closest high-status group/expert to them, without accounting for whether that group/expert is epistemically superior to other experts slightly further away in space or time. 

2a) Immodest modesty.* As a specific case/extension of this, when someone identifies an apparent expert or community of experts to defer to, they risk (incorrectly) believing that they have deference (on this particular topic) "figured out" and thus choose not to update on either object- or meta- level evidence that they did not correctly identify the relevant experts. The issue may be exacerbated beyond "normal" cases of immodesty, if there's a sufficiently high conviction that you are being epistemically modest!

3) Information lag. Obviously any information you receive is to some degree or another from the past, and has the risk of being outdated. Of course, this lag happens for all evidence you have. At the most trivial level, even sensory experience isn't really in real-time. But I think it should be reasonable to assume that attempts to read expert claims/consensus is disproportionately likely to have a significant lag problem, compared to your own present evaluations of the object-level arguments. 

4) Computational complexity in understanding the consensus. Trying to understand the academic consensus (or lack thereof) from the outside might be very difficult, to the point where establishing your own understanding from a different vantage point might be less time-consuming. Unlike 1), this presupposes that you are able to correctly understand/infer what the experts mean, just that it might not be worth the time to do so.

5) Community issues with groupthink/difficulty in separating out beliefs from action. In an ideal world, we make our independent assessments of a situation, report it to the community, in what Kant[1] calls the "public (scholarly) use of reason" and then defer to an all-things-considered epistemically modest view when we act on our beliefs in our private role as citizens.

However, in practice I think it's plausibly difficult to separate out what you personally believe from what you feel compelled to act on. One potential issue with this is that a community that's overly epistemically deferential will plausibly have less variation, and lower affordance for making mistakes.


*As a special case of that, people may be unusually bad at identifying the right experts when said experts happen to agree with their initial biases, either on the object-level or for meta-level reasons uncorrelated with truth (eg use similar diction, have similar cultural backgrounds, etc)

[1] ha!

Comment by linch on Why I'm Donating to Giving Green This Year · 2020-12-25T01:31:48.056Z · EA · GW

In that case I'm sorry I missed it!  Do you think I should delete my comment? 

Comment by linch on Why I'm Donating to Giving Green This Year · 2020-12-23T10:20:26.448Z · EA · GW

Thanks a lot for the post!

(Apologies if I missed this on a skim and/or if this comment is irrelevant) 

I think it might be helpful to put in a conflict of interest disclaimer of some sort. 

Comment by linch on Ask Rethink Priorities Anything (AMA) · 2020-12-21T18:17:23.772Z · EA · GW

Linch's mention of it below was in the context of understanding its importance rather than trying to solve it, which I guess is how I'd carve up "prioritization research".

I think what counts as prioritization vs object-level research of the form "trying to solve X" does not obviously have clean boundaries, for example a scoping paper like Concrete Problems in AI Safety is something that a) should arguably be considered prioritization research and b) is arguably better done by somebody who's familiar with (and connected in) AI. 

Comment by linch on Where I Am Donating in 2016 · 2020-12-21T17:49:09.826Z · EA · GW

Update: I've agreed to be the arbiter of the bet with Buck and Michael. My current working definition of "regularly" is something like

"A restaurant sells a product made primarily of cultured meat for at least 3+ meals/week for a continuous period of at least 4 weeks before the end of 2021"

There should probably be a stipulation on to what extent buying cultured meat is rate-limited as well.

Let me know if any EA forum reader reading this hears of something that might plausibly fulfill the bet resolution.

Comment by linch on Ask Rethink Priorities Anything (AMA) · 2020-12-21T15:32:56.807Z · EA · GW

Sure, in general feel free to assume that anything I write that's open to the public internet is fair game.

Comment by linch on Ask Rethink Priorities Anything (AMA) · 2020-12-19T23:24:52.651Z · EA · GW

Typing speed: Interesting! What is your typing speed?

Only 57.9 according to keybr. I suspect a) typing practice will be less helpful for me if my typing speed is higher (like David's) and b) my current typing speed is below average for programmers (not sure about researchers).

(It's probably  relevant/bad that my default typing system on those typing test layouts (26 characters + space only uses about 5 fingers. I think I go up to 8 on a more (normal) paragraph like this one that also uses shift/return/slash/number pad. I think if I'm focused on systematic rather than incremental changes to my typing speed I'd try to figure out how to force myself to use all 10 fingers). 

Obvious questions
Hmm I think a lot of people have motivated reasoning of the form I describe, but I don't know you well enough and I definitely don't think all people are like this.

There is certainly a danger as well of being too contrarian or self-critical. 

Have you tried calibration practice? 

Maybe also make an explicit effort to write down key beliefs and numerical probabilities (or even just words for felt senses) to record and eventually correct for overupdating on new arguments/evidence (if this is indeed your issue).

Comment by linch on [Short Version] What Helped the Voiceless? Historical Case Studies · 2020-12-17T01:40:52.313Z · EA · GW

Thanks for posting this!

By request

Full disclosure to readers: I was the one who repeatedly requested that this shorter version be its own toplevel post on the EA Forum (hope I wasn't too annoying!) ; I think forum readers + Mauricio can benefit a lot from engagement.

Some specific comments:

The framework also suggests that the following factors, when high, make it more likely that transitions to greater political inclusion will occur and persist:


  • Potential strategic alliances between an excluded and an included group (e.g. the vulnerable group, if included, would militarily/economically benefit the included)

Are you counting cases where there are intra-elite battles for power, and a certain move that franchises the voiceless also directly helps elite group A's interests at the expense of group B(eg, presumably some workers will benefit from not having to compete against slave/prison labor)?

Not sure how broad "strategic alliances" are referring to.

Comment by linch on Ask Rethink Priorities Anything (AMA) · 2020-12-16T06:40:36.443Z · EA · GW

#9 Typing speed: I think my own belief is that typing speed is probably less important than you appear to believe, but I care enough about it that I logged 53 minutes of typing practice on keybr this year (usually during moments where I'm otherwise not productive and just want to get "in flow" doing something repetitive), and I suspect I still can productively use another 3-5 hours of typing practice next year even if it trades off against deep work time (and presumably many more hours than that if it does not). 

#10 Obvious questions. I suspect that while sometimes ignoring/not noticing "obvious questions/advice" etc is coincidental unforced errors, more often than not there is some form of motivated reasoning going on behind the scenes (eg because this story will invalidate a hypothesis I'm wedded to, because it involves unpleasant tradeoffs, because some beliefs are lower prestige, because it makes the work I do seem less important, etc). I think training myself carefully to notice these things has been helpful, though I suspect I still miss a lot of obvious stuff. 

#11 Tiredness, focus, etc..I haven't figured this out yet and am keen to learn from my coworkers and others! Right now I take a lot of caffeine and I suspect if I were more careful about optimization I should be cycling drugs over a weekly basis rather than taking the same one every day (especially a drug like caffeine that has tolerance and withdrawal symptoms). 

Comment by linch on Ask Rethink Priorities Anything (AMA) · 2020-12-16T05:24:31.307Z · EA · GW

I'm not confident that this is fully outside the scope of RP, but I think backchaining-in-practice is plausibly underrated by EA/longtermism, despite a lot of chatter about it in theory. 

By backchaining in practice I mean tracing backwards fully from the world we want (eg a just, kind, safe world capable of long reflection), to specific efforts and actions individuals and small groups can do, in AI safety, biosecurity, animal welfare, movement building, etc. 

Specific things that I think will be difficult to be under RP's purview include things that require specific AI Safety or biosecurity stories, though those things plausibly have information hazards so I'd encourage people who are doing these extensive diagrams to be a) somewhat careful about information security and b) talk to the relevant people within EA (eg FHI) before creating and certainly before publishing them.

An obvious caveat here is that it's possible many such backchaining documents exist and I am unaware of them. Another caveat is that maybe backchaining is just dumb, for various epistemic reasons.

Comment by linch on Linch's Shortform · 2020-12-16T05:17:11.080Z · EA · GW

Yes that sounds right. There are also internal effects in framing/thinking/composition that by itself have flow-through effects that are plausibly >1% in expectation.

For example, more resources going into forecasting may cause other EAs to be more inclined to quantify uncertainty and focus on the quantifiable, with both potentially positive and negative flow-through effects,  more resources going into medicine- or animal welfare- heavy causes will change the gender composition of EA, and so forth. 

Comment by linch on Ask Rethink Priorities Anything (AMA) · 2020-12-16T05:02:05.155Z · EA · GW

What do you think individuals could do to become skilled in this kind of research and become competitive for these jobs?

I think this is a relatively minor thing, but trying to become close to perfectly calibrated (aka being able to put precise numbers on uncertainty) on some domains seem like a moderate-sized win, at very low cost. 

I mainly believe this because I think the costs are relatively low. My best guess is that the majority of EAs can become close to perfectly calibrated on trivia numerical questions in much less than 10 hours of deliberate practice, and my median guess is for the amount of time needed  is around 2 (eg practice here).

I want to be careful with my claims here. I think sometimes people have the impression that getting calibrated is synonymous with rationality, or intelligence, or judgement. I  think this is wrong:

  1. Concretely, I just don't think being perfectly calibrated is that big a deal. My guess is that going from median-EA levels of general calibration to perfect calibration on trivia questions is an improvement in good research/thinking by 0.2%-1%. I will be surprised if somebody becomes a better researcher by 5% via these exercises, and very surprised  if they improve by 30%.
  2. In forecasting/modeling, the main quantifiable metrics include both a) calibration (roughly speaking, being able to quantify your uncertainty) and b) discrimination (roughly speaking, how often you're right). In the vast majority of cases, calibration is just much less important than discrimination. 
  3. There are generalizability issues with generalizing from good calibration on trivia questions to good calibration overall. The latter is likely to be much harder to train precisely, or even precisely quantify (though I'm reasonably confident that going from poor calibration on trivia to perfect calibration should generalize somewhat, Dave Bernard might have clearer thoughts on this)
  4. I think calibration matters more for generalist/secondary research (much of what RP does) than for things that either a) require relatively narrow domain expertise, like ML-heavy AI Safety research or biology-heavy biosecurity work, or b)  require unusually novel thinking/insight (like much of crucial considerations work). 

Nonetheless, I'm a strong advocate for calibration practice because I think the first hour or two of practice will pay off by 1-2 orders of magnitude over your lifetime, and it's hard to identify easy wins like that (I suspect even exercise has a less favorable cost-benefits ratio, though of course it's much easier to scale).

Comment by linch on Ask Rethink Priorities Anything (AMA) · 2020-12-16T03:23:27.338Z · EA · GW

I did internal modeling/forecasting for our fundraising figures, and at least on the first pass it looked like our longtermist work was more likely to be funding constrained than our other priority cause areas, at least if "funding constrained" is narrowly defined as "what's the probability that we do not raise all the money that we'd like for all planned operations to run smoothly."

My main reasoning was somewhat outside-viewy and focused on general uncertainty: our longtermist team is new, and relative to other of Rethink's cause areas, less well-established with less of a track record of either a) prior funding, b) public work other than Luisa's nuclear risk work, or c) a well-vetted research plan. So I'm just generally unsure of these things.

Three major caveats:

1. I did those forecasts in late October and now I think my original figures were too pessimistic. 

2. Another caveat is that my predictions were more a reflection of my own uncertainty than a lack of inside view confidence in the team. For context, my 5th-95th percentile credible interval spanned ~an order of magnitude across all cause areas.

3. When making the original numbers, I incorporated but plausibly substantially underrated the degree that  the forecasts will change and not just reflect reality. For example, Peter and Marcus may have prioritized different decisions accordingly due to my numbers, or this comment may affect other people's decisions.

Comment by linch on Ask Rethink Priorities Anything (AMA) · 2020-12-16T02:51:12.719Z · EA · GW

(Speaking for myself and not others on the team, etc)

 At a very high level, I think I have mostly "mainstream longtermist EA" views here, and my current best guess would be that AI Safety, existential biosecurity, and cause prioritization (broadly construed) are the highest EV efforts to work on overall, object-level. 

This does not necessarily mean that marginal progress on these things are the best use of additional resources, or that they are the most cost-effective efforts to work on, of course.

Comment by linch on Jamie_Harris's Shortform · 2020-12-14T20:43:59.698Z · EA · GW

In a short Google Form, posted on the Effective Altruism Researchers and EA Academia Facebook groups,  I provided the above paragraph and then asked: "If, as well as an undergraduate/bachelor's degree, they start their research career at EA nonprofits with a master's degree in a relevant field, how many "units" of impact do you expect that they would produce each year for the first ~10 years of work?"* The average response, from the 8 respondents, was 1.7.

The mechanism may not be causal. If you're conditioning on type of person who can get accepted into graduate programs + get funding + manage to stick with a PhD program, you are implicitly drawing on a very different pool of people than if you don't condition on this.

Comment by linch on Idea: the "woketionary" · 2020-12-14T04:50:31.226Z · EA · GW

I think it is somewhat unlikely that this will meet the fairly high EA bar for being a worthwhile donation or volunteering opportunity. 

Comment by linch on Long-Term Future Fund: Ask Us Anything! · 2020-12-10T08:09:39.595Z · EA · GW

Compounding this problem, aside from that one sentence the fund page (even after it has been edited for clarity) makes it sound like AI and pandemics are prioritized similarly, and not that far above other LT cause areas. I believe the LTFF has only made a few grants related to pandemics, and would guess that AI has received at least 10 times as much funding

Adam has mentioned elsewhere here that he will prefer making more biosecurity grants. An interesting question here is how much the messaging should be descriptive of past donations, vs aspirational of where they want to donate more to in the future.

Comment by linch on My mistakes on the path to impact · 2020-12-10T02:11:15.348Z · EA · GW

(My best guess is that the average EA defers too much rather than too little. This and other comments on deference is to address specific points made, rather than to push any particular general takes).

Maybe that's because startups are a more heavy-tailed domain than altruism, and one where conformity is more harmful

I think this is part of the reason. A plausibly bigger reason is that VC funding can't result in heavy left-tails. Or rather, left-tails in VC funding are very rarely internalized. Concretely, if you pick your favorite example of "terrible startup for the future of sentient beings," the VCs in question very rarely get in trouble, and approximately never get punished proportional to the counterfactual harm of their investments. VC funding can be negative for the VC beyond the opportunity cost of money (eg via reputational risk or whatever), but the punishment is quite low relative to the utility costs. 

Obviously optimizing for increasing variance is a better deal when you clip the left tail, and optimizing for reducing variance is a better deal when you clip the right tail.

(I also independently think that heavy left tails in the utilitarian sense are probably less common in VC funding than in EA, but I think this is not necessary for my argument to go through).

Comment by linch on Linch's Shortform · 2020-12-10T01:04:25.724Z · EA · GW

Yes, though I'm strictly more confident about absolute value than the change being  positive (So more resources R going into Y can also eventually lead to less resources going into X, within about R/10^2).

Comment by linch on Best Consequentialists in Poli Sci #1 : Are Parliaments Better? · 2020-12-08T22:24:24.475Z · EA · GW

It seems like whether we should push for something to be included in a constitution (or any other significant intervention) depends not only on whether the change is good, but also how large the expected effect size is. Am I reading the tables correctly that the effect size of Parliamentarism, while robustly positive, is tiny relative to other factors that contribute to the various R^2s in the models? 

Comment by linch on Long-Term Future Fund: Ask Us Anything! · 2020-12-08T20:00:09.293Z · EA · GW

Thanks a lot, really appreciate your thoughts here!

Comment by linch on My mistakes on the path to impact · 2020-12-08T18:24:22.749Z · EA · GW

If 100 forecasters (who I roughly respect) look at the likelihood of a future event and think it's ~10% likely, and I look at the same question and think it's ~33% likely, I think I will be incorrect in  my private use of reason for my all-things-considered-view to not update  somewhat downwards from 33%. 

I think this continues to be true even if we all in theory have access to the same public evidence, etc. 

Now, it does depend a bit on the context of what this information is for. For example if I'm asked to give my perspective on a group forecast (and I know that the other 100 forecasters' predictions will be included anyway), I think it probably makes sense for me to continue to publicly  provide ~33% for that question to prevent double-counting and groupthink. 

But I think it will be wrong for me to believe 33%, and even more so, wrong to say 33% in a context where somebody else doesn't have access to the 100 other forecasters. 

An additional general concern here to me is computational capacity/kindness-- sometimes (often) I just don't have enough time to evaluate all the object-level arguments! You can maybe argue that until I evaluate all the object-level arguments, I shouldn't act, yet in practice I feel like I act with lots of uncertainty* all the time!

One disagreement I have with Max is whether someone should defer is contingent upon the importance of a decision. I think this begs the question in that it pre-assumes that deference lead to the best outcomes. 

Instead, I think you should act such that you all-things-considered-view is that you're making the best decision. I do think that for many decisions (with the possible exception of creative work), some level of deference leads to better outcomes than zero deference at all, but I don't think it's unusually true for important decisions except inasmuch as a) the benefits (and also costs!) of deference are scaled accordingly and b) more people are likely to have thought about important decisions.

* Narrow, personal, example that's basically unrelated to EA: I brush my teeth with fluoride toothpaste. I don't floss. Why? Cochrane review was fairly equivocal about flossing and fairly certain about toothbrushing. Maybe it'd be more principled if I looked at the data myself and performed my own meta-analysis on the data, or perhaps self-experimented like Gwern, to decide what dental hygiene activities I should take. But in practice I feel like it's a reasonable decision procedure to just defer to Cochrane review on the empirical facts of the matter, and apply my own value judgments on what activities to take given the facts available.

Comment by linch on Linch's Shortform · 2020-12-08T15:28:31.780Z · EA · GW

On the forum, it appears to have gotten harder for me to do multiple quote blocks in the same comment. I now often have to edit a post multiple times so quoted sentences are correctly in quote blocks, and unquoted sections are not. Whereas in the past I do not recall having this problem?

Comment by linch on Linch's Shortform · 2020-12-08T15:21:27.864Z · EA · GW

By "meta concerns", do you mean stuff like base rate of interventions, risk of being wildly wrong, methodological errors/biases, etc.?

Hmm I think those are concerns too, but I guess I was primarily thinking about meta-EA concerns like whether an intervention increases or decreases EA prestige, willingness of new talent to work on EA organizations, etc.

Also, did you mean that these dominate when object-level impacts are big enough?

No. Sorry I was maybe being a bit confusing with my language. I mean to say that when comparing two interventions, the meta-level impacts of the less effective intervention will dominate if you believe the object-level impact of the less effective intervention is sufficiently small.

 Consider two altruistic interventions, direct AI Safety research and forecasting. Suppose that you did the analysis and think the object-level impact of AI Safety research is X (very high) and the impact of forecasting  is only 0.0001X.

 (This is just an example. I do not believe that the value of forecasting is 10,000 times lower than AI Safety research). 

I think it will then be wrong to think that the all-things-considered value of an EA doing forecasting is 10,000 times lower than the value of an EA doing direct AI Safety research, if for no other reason than because EAs doing forecasting has knock-on effects on EAs doing AI Safety. 

If the object-level impacts of the less effective intervention are big enough, then it's less obvious that the meta-level impacts will dominate. If your analysis instead gave a value of forecasting as 3x less impactful than AIS research, then I have to actually present a fairly strong argument for why the meta-level impacts may still dominate, whereas I think it's much more self-evident at the 10,000x difference level. 

Let me know if this is still unclear, happy to expand. 

Oh, also a lot of my concerns (in this particular regard) mirror Brian Tomasik's, so maybe it'd be easier to just read his post.

Comment by linch on Linch's Shortform · 2020-12-08T08:00:25.063Z · EA · GW

I continue to be fairly skeptical that the all-things-considered impact of EA altruistic interventions differ by multiple ( say >2) orders of magnitude ex ante (though I think it's plausible ex post). My main crux here is that I believe general meta concerns start dominating once the object-level impacts are small enough.

This is all in terms of absolute value of impact. I think it's quite possible that some interventions have large (or moderately sized) negative impact, and I don't know how the language of impact in terms of multiplication best deals with this.

Comment by linch on How to Fix Private Prisons and Immigration · 2020-12-08T07:55:44.679Z · EA · GW

Thanks for your engagement!

And denying the trade-off doesn't mean the inmate is not looked after either

Agreed, but at least in theory, a model that takes into account inmate's welfare at the proper level will, all else being equal, do better under utilitarian lights than a model that does not take into account inmate welfare. 

This may be an obvious point, but I've made this same mistake ~4 years ago when discussing a different topic (animal testing), so I think it's worth flagging explicitly. 

I'm not 100 percent set on the exact funding function. I might change my mind in the future.


Please feel free to edit the post if you do! I worry that many posts (my own included) on the internet are stale, and we don't currently have a protocol in place for declaring things to be outdated.

Comment by linch on Linch's Shortform · 2020-12-08T00:45:14.727Z · EA · GW

In the Precipice, Toby Ord very roughly estimates that the risk of extinction from supervolcanoes this century is 1/10,000 (as opposed to 1/10,000 from natural pandemics, 1/1,000 from nuclear war, 1/30 from engineered pandemics and 1/10 from AGI). Should more longtermist resources be put into measuring and averting the worst consequences of supervolcanic eruption?

More concretely, I know a PhD geologist who's interested in doing an EA/longtermist career and is currently thinking of re-skilling for AI policy. Given that (AFAICT) literally zero people in our community currently works on supervolcanoes, should I instead convince him to investigate supervolcanoes at least for a few weeks/months? 

Comment by linch on How to Fix Private Prisons and Immigration · 2020-12-08T00:05:35.598Z · EA · GW

Societal contribution and a person's value are different things: A person who lives separately from society has value. But I don't know how to construct a system that incorporates that value.

Possibly a tangent, but I think it's maybe relevant that QALYs do not have that problem.