EA Forum Reader

Thanks. I agree that we should have non-infinitesimal credence that physics is wrong, but to change the conclusion, we would need to "insist that modern physics is incorrect in very specific ways." Given the strength of evidence about the existence of many of the limits, regardless of their actual form or value, that is a higher bar. I also advise looking closely at the discussion of the "Pessimistic Meta-induction," and why we think that it's reasonable to be at least incredibly confident that these limits exist.

That doesn't guarantee their existence. But after accepting a non-zero credence in those specific types of incorrect theory, we need to pin our hopes for infinite value on those specific occurrences; we would need to maximize expected value conditional on that very small probability in order to find infinite value, and neglect the very large but finite value we are nearly certain exists in the physical universe. That seems difficult to me.

Thanks, and thanks for posting this both places. I've responded on the lesswrong post [LW(p) · GW(p)], and I'm going to try to keep only one thread going, given my finite capacity to track things :)

Well said. I agree that that is a path to impact for the sort of work QRI is doing, it just seems lower-priority to me than other things like working on AI alignment or AI governance.  Not to mention the tractability / neglectedness concerns (philosophy is famously intractable, and there's an entire academic discipline for it already)

Hi Daniel,

Thanks for the reply! I am a bit surprised at this:

Getting more clarity on emotional valence does not seem particularly high-leverage to me. What's the argument that it is?

The quippy version is that, if we’re EAs trying to maximize utility, and we don’t have a good understanding of what utility is, more clarity on such concepts seems obviously insanely high-leverage. I’ve written about specific relevant to FAI here: https://opentheory.net/2015/09/fai_and_valence/ Relevance to building a better QALY here: https://opentheory.net/2015/06/effective-altruism-and-building-a-better-qaly/ And I discuss object-level considerations on how better understanding of emotional valence could lead to novel therapies for well-being here: https://opentheory.net/2018/08/a-future-for-neuroscience/ https://opentheory.net/2019/11/neural-annealing-toward-a-neural-theory-of-everything/ On mobile, pardon the formatting.

Your points about sufficiently advanced AIs obsoleting human philosophers are well-taken, though I would touch back on my concern that we won’t have particular clarity on philosophical path-dependencies in AI development without doing some of the initial work ourselves, and these questions could end up being incredibly significant for our long-term trajectory — I gave a talk about this for MCS that I’ll try to get transcribed (in the meantime I can share my slides if you’re interested). I’d also be curious to flip your criticism and ping your models for a positive model for directing EA donations — is the implication that there are no good places to donate to, or that narrow-sense AI safety is the only useful place for donations? What do you think the highest-leverage questions to work on are? And how big are your ‘metaphysical uncertainty error bars’? What sorts of work would shrink these bars?

What are the community's thoughts on making the application super long so that only the most interested people apply (and basically accept everyone who applies)? Would this be considered selective in the same way as rejecting people?

Great post, and interesting and surprising result.

An obvious alternative selection criterion would be something like “how good would it be if this person got really into EA”; I wonder if you would be any better at predicting that. This one takes longer to get feedback on, unfortunately.

I think I'd just note that the post, in my opinion, helps combat some of these issues. For instance it suggests that autistic people are able to learn how to interact with neurotypical people successfully, given sufficient effort--ie, the "mask".

Definitely agree with this, as someone currently at UChicago! The Center for Radical Innovation for Social Change (RISC) recently put out a call for animal welfare proposals and Steve Levitt has connections to Schmidt Futures (an EA-adjacent philanthropic initiative), so that could be a promising place to start.

Thank you for this - very well-organized, clear, & relatable as someone struggling to figure out their first foray into the professional world.

Definitely agree that having more posts like this would be helpful considering the selectivity + relatively low supply of formal EA mentorship programs.

That's fair. My understanding though is that management training doesn't seem very useful in general, implying that either the things they are teaching aren't very useful or people aren't very good at filtering to find the parts that are useful to them.