EA Forum Reader

The Center for Human-Compatible Artificial Intelligence (CHAI) at UC Berkeley is in need of a new Operations Assistant. The purpose of this position is to ensure smooth functioning of the CHAI organization by working with CHAI Assistant Director Martin Fukui and the rest of the team with a variety of tasks, described in the link below under 'Responsibilities'. Some of this work can be done remotely, but some work must be done on campus, primarily at Berkeley Way West.

https://existence.org/jobs/chai-operations-assistant

for more information about the role, please contact andrew@humancompatible.ai 

Even without considering that, if we stay at ~140 million births per year, in 800 years 50% of all humans will have been born in our future.
And in ~7 millennia 90% of all humans will have been born in our future.

Thanks, Thomas! I think that's a fair criticism. What I was trying to get at with "some actions that fall within the category of skill building would be extremely useful for some and trivial for others" is that to a certain extent, it's up to the individual to define what skill building is for them. I'm amenable to the fact that maybe these definitions weren't lenient enough to a range of career paths. I do worry, however, that too many individuals are going into ops/community building without building up other aptitudes. 

Referring to your last paragraph, I think it'd be better if a group organizer could get those same skills in something a bit more catered to a prospective career path, although maybe this is unrealistic. This also doesn't hold if the individual's career interest is in ops. 

Thanks, Akash! I haven't thought a lot about how this might apply to larger-scale events like retreats, but this makes a lot of sense to me. Somewhat unrelatedly, I think it'd be nice to have a catalog / google doc of workshops that are all about skill-building (or "learning together or working on interesting unsolved problems together." as you put it).  I felt like the Bright Futures Retreat had a lot of good examples of this. University group organizers then have a good reference of what types of events are useful, and they can either 1) use the catalog as inspiration for spin-off workshops 2) host workshops from the catalog as one-off events during term, or 3) compile their favorites for a retreat. This would likely save them a lot of time and diminish the amount of ops-related work they do.  

Because it's possible that even in unstable, diverse futures, catastrophe can be avoided. As to the long-term future after the Singularity, that's a question we will deal with it when we get there

Meta: I'm currently writing up a post with a fully-fleshed-out account of deference. If you'd like to drop this thread and engage with that when it comes out (or drop this thread without engaging with that), feel free; I expect it to be easier to debate when I've described the position I'm defending in more detail.

I'll note that most of this seems unrelated to my original claim, which was just "deference* seems important for people making decisions now, even if it isn't very important in practice for researchers", in contradiction to a sentence on your top-level comment. Do you now agree with that claim?

I always agreed with this claim; my point was that the type of deference which is important for people making decisions now should not be very sensitive to the "specific credences" of the people you're deferring to. You were arguing above that the difference between your and Eliezer's views makes much more than a 2x difference; do you now agree that, on my account of deference, a big change in the deference-weight you assign to Eliezer plausibly leads to a much smaller change in your policy from the perspective of other worldviews, because the Eliezer-worldview trades off influence over most parts of the policy for influence over the parts that the Eliezer-worldview thinks are crucial and other policies don't?

individual proxies and my thoughts on them

This is helpful, thanks. I of course agree that we should consider both correlations with impact and ease of evaluation; I'm talking so much about the former because not noticing this seems like the default mistake that people make when thinking about epistemic modesty. Relatedly, I think my biggest points of disagreement with your list are:

1. I think calibrated credences are badly-correlated with expected future impact, because:
a) Overconfidence is just so common, and top experts are often really miscalibrated even when they have really good models of their field
b) The people who are best at having impact have  goals other than sounding calibrated - e.g. convincing people to work with them, fighting social pressure towards conformity, etc. By contrast, the people who are best at being calibrated are likely the ones who are always stating their all-things-considered views, and who therefore may have very poor object-level models. This is particularly worrying when we're trying to infer credences from tone - e.g. it's hard to distinguish the hypotheses "Eliezer's inside views are less calibrated than other peoples" and "Eliezer always speaks based on his inside-view credences, whereas other people usually speak based on their all-things-considered credences".
c) I think that "directionally correct beliefs" are much better-correlated, and not that much harder to evaluate, and so credences are especially unhelpful by comparison to those (like, 2/10 before conditioning on directional correctness, and 1/10 after, whereas directional correctness is like 3/10).

2. I think coherence is very well-correlated with expected future impact (like, 5/10), because impact is heavy-tailed and the biggest sources of impact often require strong, coherent views. I don't think it's that hard to evaluate in hindsight, because the more coherent a view is, the more easily it's falsified by history.

3. I think "hypothetical impact of past policies" is not that hard to evaluate.  E.g. in Eliezer's case the main impact is "people do a bunch of technical alignment work much earlier", which I think we both agree is robustly good.

Good catch! I've just edited to correct this, thanks.

I think that retroactive Impact Markets may be a net negative for many x-risk projects, however, I also think that in general Impact Markets may significantly reduce x-risk.

I think you have to bear in mind that if this project is highly successful, it has the potential to create a revolution in the funding of public goods. If humanity achieves much better funding and incentive mechanisms for public goods, this could create a massive increase in the efficiency of philanthropy.

It is hard to know how beneficial such a system would be, but it is not hard to see how mutilpying the effectiveness of philanthropy and public good provision could make society function much better by improving education, coordination, mental health, moral development, health, etc., and increases in these public goods could broadly improve humanity’s ability to confront many x-risks.

I think it may make sense for Impact Markets to find ways if limiting or excluding x-risk projects, but I think abandoning Impact Markets altogether would be a mistake, and considering their massive upsides I cannot say I agree that they are net negative in expectation, even without excluding x-risk projects.

I'm not sure why this should be reassuring. It doesn't sound clearly good to me. In fact, it sounds pretty controversial. 

Basically, yes. Assuming civilization survives the Singularity, existential risks are effectively zero thanks to the fact that it's almost impossible to destroy an interstellar civilization.