Posts

Long Term Future Fund application is closing this Friday (October 11th) 2019-10-10T00:43:28.728Z · score: 13 (3 votes)
Long-Term Future Fund: August 2019 grant recommendations 2019-10-03T18:46:40.813Z · score: 79 (35 votes)
Survival and Flourishing Fund Applications closing in 3 days 2019-10-02T00:13:32.289Z · score: 11 (6 votes)
Survival and Flourishing Fund grant applications open until October 4th ($1MM-$2MM planned for dispersal) 2019-09-09T04:14:02.083Z · score: 29 (10 votes)
Integrity and accountability are core parts of rationality [LW-Crosspost] 2019-07-23T00:14:56.417Z · score: 52 (20 votes)
Long Term Future Fund and EA Meta Fund applications open until June 28th 2019-06-10T20:37:51.048Z · score: 60 (23 votes)
Long-Term Future Fund: April 2019 grant recommendations 2019-04-23T07:00:00.000Z · score: 137 (73 votes)
Major Donation: Long Term Future Fund Application Extended 1 Week 2019-02-16T23:28:45.666Z · score: 41 (19 votes)
EA Funds: Long-Term Future fund is open to applications until Feb. 7th 2019-01-17T20:25:29.163Z · score: 19 (13 votes)
Long Term Future Fund: November grant decisions 2018-12-02T00:26:50.849Z · score: 35 (29 votes)
EA Funds: Long-Term Future fund is open to applications until November 24th (this Saturday) 2018-11-21T03:41:38.850Z · score: 21 (11 votes)

Comments

Comment by habryka on Which Community Building Projects Get Funded? · 2019-11-15T19:25:17.049Z · score: 19 (7 votes) · EA · GW

I am confused by this. I consider it a key responsibility of the meta fund to independently fund community building grants, and this is a major update downwards on the value of the meta fund for me.

I would strongly urge you to consider investing more into evaluating and vetting community-building grants. I don't think it's healthy for CEA to be the only funder in this space.

Comment by habryka on Which Community Building Projects Get Funded? · 2019-11-14T20:32:12.682Z · score: 29 (16 votes) · EA · GW
The largest grant (accounting for nearly a quarter of all EA Community funding) went to a project run by CEA’s former Director of Strategy.

Just as a more transparent reference, this refers to a grant to me and the LessWrong team to build the new LessWrong and EA Forum platforms. It's correct that we are located in the Bay Area, but importantly LessWrong and the EA Forum itself are not specific to a geographic location, and I actually think of them as key parts of having more distributed community-building infrastructure.

It's also important because there has been very little investment in Bay-Area EA community building in the past few years, even though it is one of the biggest EA hubs, and I don't want people to think this grant helped much with that. We do sometimes run events, but we are first and foremost an online-community building organization.

I also find it important to point out that I was the director of strategy for CEA US, not all of CEA. At the time I was at CEA the organization was much less integrated and I think CEA at the time was better modeled as two organizations, both of which were much smaller than CEA is now.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-25T21:52:29.652Z · score: 4 (2 votes) · EA · GW

Thanks for pinging me!

I am still pretty swamped (being in the middle of both another LTFF grant round and the SFF grant round), and since I think a proper response to the above requires writing quite a bit of text, it will probably be another two weeks or so.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-15T16:31:55.410Z · score: 5 (3 votes) · EA · GW

I think this is a good question. I am currently on a team retreat, so likely won't get to this until next week (and maybe not then because I will likely be busy catching up with stuff). If I haven't responded in 10 days, please feel free to ping me.

Comment by habryka on Publication of Stuart Russell’s new book on AI safety - reviews needed · 2019-10-12T19:39:38.932Z · score: 2 (1 votes) · EA · GW

Yes, that is correct. Audible and Amazon have country settings that you have to change when you move. This article should cover how to change the amazon one:

https://smile.amazon.com/gp/help/customer/display.html?nodeId=201248840&sa-no-redirect=1

Comment by habryka on Stefan_Schubert's Shortform · 2019-10-12T19:37:42.544Z · score: 9 (3 votes) · EA · GW

This seems about a factor of 2 lower than I expected. My guess would be that this just includes the actual cost of fixing formatting errors, not the cost of fitting your ideas to fit the formatting at all (i.e. having to write all the different sections, even when it doesn't make sense, or being forced to use LaTeX in the first place).

(Note: I did not yet get around to reading the paper, so this is just a first impression, as well as registering a prediction)

Comment by habryka on Publication of Stuart Russell’s new book on AI safety - reviews needed · 2019-10-12T06:40:29.277Z · score: 3 (2 votes) · EA · GW

I had the same problem, for me it turned out to be caused by Amazon and audible separately keeping track of the country in which my account was registered. I can imagine that you are running into the same problem, since you relatively recently moved to the UK (at least if I remember correctly).

Comment by habryka on X-risk dollars -> Andrew Yang? · 2019-10-12T00:39:28.549Z · score: 8 (2 votes) · EA · GW

My (relatively weak and mostly intuitive) sense is that automation of labor and surrounding legislation has become a pretty polarized issue on which rational analysis has become quite difficult, so I don't think this seems like a good counterexample.

Comment by habryka on X-risk dollars -> Andrew Yang? · 2019-10-11T22:46:41.576Z · score: 6 (4 votes) · EA · GW
A cynical interpretation of the tweet is that we learned that Bostrom has one (maxed out) donor who likes Bostrom.

Presumably should say "that Yang has one (maxed out) donor who likes Bostrom"

Comment by habryka on X-risk dollars -> Andrew Yang? · 2019-10-11T21:53:18.739Z · score: 20 (6 votes) · EA · GW

Making AI Alignment into a highly polarized partisan issue would be an obvious one.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-10T23:53:41.695Z · score: 16 (5 votes) · EA · GW

This is great, and I think these counterpoints are valuable to read for anyone interested in this topic. I disagree with sections of this (and sometimes agree but just think the balance of considerations plays out differently), and will try to find the time to respond to this in more detail in at least the coming weeks.

Comment by habryka on JP's Shortform · 2019-10-10T21:03:36.694Z · score: 2 (1 votes) · EA · GW

Oh, interesting. LessWrong always had that, and I never even thought about that maybe being a configuration difference between the two sites.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-10T16:44:33.896Z · score: 12 (6 votes) · EA · GW

My sense is mostly (1), with maybe some additional disagreement over what online contributions are actually a sign of competence. But usually I am quite excited when an active online contributor applies.

I share your perspective that I am most excited about people who participate in AI Alignment discussion online, but we’ve received relatively few applications from people in that reference class.

Some of the people we’ve given grants to, were the result of Alex Zhu doing a lot of networking with people who are interested in AI alignment, which tends to select on some slightly different things, but given the lack of applications from people with a history of contributing online, that still seems pretty good to me.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-10T05:09:19.370Z · score: 21 (6 votes) · EA · GW

This is a bit hard to answer for me, because there are three grants that I was quite excited about that we didn't end up making, that I think were more valuable than many of the grants we did end up making, so maybe a different grant member should answer this question.

If I exclude those three grants, I think there were grants we didn't fund that are about as good as the ones we funded, at least from my personal perspective.

It's harder for me to give an answer "from the perspective of the whole fund", but I would still be surprised if the next grant would have a marginal cost-effectiveness of less than 90% of the marginal grant this round, though I think these things tend to be pretty high variance, so probably only 60% of the average grant this round.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-09T00:35:41.761Z · score: 6 (5 votes) · EA · GW

I haven't looked super much into the literature on this so I might be wrong, my sense was that it was more of a case of "lots of therapeutic techniques share a lot of structure, and Gendlin formalized it into a specific technique, but a lot of them share a lot of structure with what Gendlin is doing", which makes sense, because that's how focusing was developed. From the Wikipedia article:

Gendlin developed a way of measuring the extent to which an individual refers to a felt sense; and he found in a series of studies that therapy clients who have positive outcomes do much more of this. He then developed a way to teach people to refer to their felt sense, so clients could do better in therapy. This training is called 'Focusing'. Further research showed that Focusing can be used outside therapy to address a variety of issues.

The thing that made me more comfortable saying the above was that Gendlin's goal (judging from the focusing book I read and the audiobook I listened to) seems to have been in significant parts a study into "what makes existing therapeutic techniques work", instead of "let's develop a new technique that will revolutionize therapy", so even if a school of therapy isn't downstream of Gendlin, you expect a good fraction to still have focusing-like things in them, since Gendlin seemed to be more interested in refining techniques instead of revolutionizing them.

I do agree that I should probably stop using words like "significant fraction". I intended to mean something like 20%-30% of therapy sessions will likely include something that is pretty similar to focusing, even if it isn't exactly called that, which still seems roughly right to me and matches with my own experience of therapy with a practitioner who specialized in CBT and some trauma-specific therapies, but our actual sessions weren't really utilizing either of those schools and were basically just focusing sessions, which to that therapist seemed like the natural thing to do in the absence of following a more specific procedure.

Some of my impression here also comes from two textbooks I read on therapy whose names I currently forgot, both of which were mostly school-independent and seemed to emphasize a lot of focusing-like techniques.

However, I don't have super strong models here, and a significant fraction of my models are downstream of Gendlin's own writing (who as I said seems to describe focusing more as "the thing that makes most type of therapy work"), so I am pretty open to being convinced I am wrong about this. I can particularly imagine that Freudian approaches could do less focusing, since I've basically not interacted with anything in that space and feel kinda averse to it, so I am kind of blind to a significant fraction of the therapy landscape.

Comment by habryka on JP's Shortform · 2019-10-08T17:13:43.837Z · score: 3 (2 votes) · EA · GW

Making it unchangeable also seems reasonable to me, or at least making it so that you can no longer strong-upvote your own comments.

Strong-upvoting your own posts seems reasonable to me (and is also the current default behavior)

Comment by habryka on JP's Shortform · 2019-10-08T17:11:25.078Z · score: 2 (1 votes) · EA · GW

Yeah, you can add lots of additional fields. It also has like 100 options for changing the algorithm (including things like changing the importance of spelling errors in search, and its eagerness to correct them), so playing around with that might make sense.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-08T03:21:19.544Z · score: 6 (4 votes) · EA · GW

Thank you! :)

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-08T02:57:24.368Z · score: 24 (7 votes) · EA · GW

I don't think there is a single link, though most of the concepts have a pretty good canonical resource. I do think it usually takes quite a bit of text to convey each of those concepts, so I don't think creating a single written reference is easily feasible, unless someone wants to produce multiple books worth of content (I've historically been impressed with how much content you can convey in a 1.5 hour long class, often 10 blog posts worth, or about half of a book).

I don't think I have the time to compile a full list of resources for each of these concepts, but I will share the top things that come to mind.

  • Units of Exchange: I think microeconomics classes do a pretty good job of this, though are usually a bit abstract. A lot of writing of Scott Alexander gets at this, with the best introduction probably being his "Efficient Charity: Do unto others..."
  • Inner Simulator: Covered pretty well by Thinking: Fast and Slow by Daniel Kahneman
  • Trigger-Action Planning: Also covered pretty well by Thinking Fast and Slow, though, with some Getting Things Done thrown into it
  • Goal Factoring: I don't actually know a good introduction to this, alas.
  • Understanding Shoulds: Mindingourway.com's "Replacing Guilt" series
  • Focusing: The best introduction into this is Gendlin's audiobook, which I highly recommend and is relatively short
  • Systemization: As mentioned, Getting Things Done is the best introduction into this topic
  • Double Crux: I think Duncan Sabien's introduction for this is probably the best one
Comment by habryka on JP's Shortform · 2019-10-07T23:01:01.917Z · score: 2 (1 votes) · EA · GW

(Edit: This was written before Khorton edited a concrete example into their comment)

Interesting. I haven't had many issues with the search. I mostly just wanted it to have more options that I can tweak (like restricting it to a specific time period and author). If you know of any site (that isn't a major search engine provider) that has search that does better here, I would be curious to look into what technology they use (we use Algolia, which seems to be one of the most popular search providers out there, and people seem to generally be happy with it). It might also be an issue of configuration.

Comment by habryka on JP's Shortform · 2019-10-07T22:30:05.524Z · score: 2 (1 votes) · EA · GW

Curious what the problem with the current search engine is? Agree that it's important to be able to find forum posts via Google, which is currently an EA Forum specific issue, but improvements to the search likely also affect LessWrong, so I am curious in getting more detail on that.

Comment by habryka on [Link] What opinions do you hold that you would be reluctant to express in front of a group of effective altruists? Anonymous form. · 2019-10-07T20:52:17.217Z · score: 10 (7 votes) · EA · GW

(This is publicly available information, so I hope it's fine if I share this. I noticed some people had downvoted this comment earlier on, so I am a bit hesitant, but after thinking more about it, I can't think of any particular reason why this question should go unanswered.)

Halstead works at Founders Pledge.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-07T18:08:47.028Z · score: 19 (6 votes) · EA · GW

(Reply written after the paragraph was added above)

Thanks for the elaboration! Some quick thoughts:

qualitatively describe what you're trying to teach, and why you think this is a good idea; monitor and publish data such as number of workshops run, attendance etc., without narrowly optimizing for any of these

I think CFAR has done at least everything on this list of examples. Which you might already be aware of, but wanted to make sure is common knowledge. There are a significant number of posts trying to explain CFAR at a high-level, and the example workshop schedule summarizes all the classes at a high-level. CFAR has also published the number of workshops they've run and their total attendance in their impact reports and on their homepage (currently listing 1045 alumni). Obviously I don't think that alone is sufficient, but it seemed plausible that a reader might walk away thinking that CFAR hadn't done any of the things you list.

disagree with your implied criterion of using something like "quality-weighted sum of generated research" is an appropriate main criterion for assessing the education system, and thus by extension disagree with the emphasis on right-tail outcomes when evaluating the public education system as a whole

I think there is some truth to this interpretation, but I think it's overall still wrong enough that I would want to correct it. I think the education system has many goals, and I don't think I would summarize it's primary output as "quality-weighted sum of generated research". I don't think going into my models of the education system here is going to be super valuable, though happy to do that at some other point if anyone is interested in them. My primary point was that optimizing for legibility clearly has had large effects on educational institutions, in ways that would at least be harmful to CFAR if affected in the same way (another good example here might be top universities and the competition for getting into all the top 10 ranking, though I am less confident of the dynamics of that effects).


Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-07T17:54:59.798Z · score: 15 (8 votes) · EA · GW

For whatever it's worth, this seems right to me, and I do want to make sure that people know that I do think CFAR should try to be more legible at the margin

I mentioned this in my writeup above:

I think that CFAR is still likely optimizing too little towards legibility, compared to what I think would be ideal for it. Being legible allows an organization to be more confident that its work is having real effects, because it acquires evidence that holds up to a variety of different viewpoints.

I do think the question of what the correct outcome measurements for an impact evaluation would be is non-trivial, and would be interested in whether people have any good ideas for good outcome measurements.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-07T17:49:55.288Z · score: 13 (8 votes) · EA · GW

(Edit the below was written before Max edited the second paragraph into his comment)

Seems good! I actually think considerations around legibility are quite important and where I expect a good amount of intellectual progress to be made by talking to each other, so I would like to see your perspective written up and engage with it.

I also want to make sure that it's clear that I do think CFAR should be more legible and transparent (as I said in the writeup above). I have some concerns with organizations trying to be overly legible, but I think we both agree that at the current margin it would be better for CFAR to optimize more for legibility.

(I've sadly had every single comment of mine on this thread strong-downvoted by at least one person, and often multiple people. My sense is that CFAR is a pretty polarizing topic, which I think makes it particularly important to have this conversation, but seems to also cause some unfortunate voting patterns that feel somewhat stressful to deal with.)

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-07T17:32:18.158Z · score: 21 (9 votes) · EA · GW

I would be open to helping run such an RCT, and by default would expect the written material without further assistance to have relatively little impact.

I also think that for many people asking them to read the related online material will have a much lower completion rate than going to a workshop, and figuring out how to deal with that would be a major uncertainty in the design of the RCT. I have many friends that I tried to get to desperately read the material that explains the above core concepts, sadly without success, who finally got interested enough into all of the above after attending a CFAR workshop.

In my last 5 years of working in EA and the rationality community, I have repeatedly been surprised by the degree to which even very established EAs have not read almost any introductions to the material I outlined above, and where the CFAR workshop was their first introduction into the material. This includes large parts of the staff at CEA, as well as many core group organizers I've met.

I don't expect CFAR putting out online material to help much with this, since roughly the same holds true for 80k material, and a lot of the concepts above actually already have good written explanations to them.

You seem to be very optimistic about getting people to read written content, whereas my experience has been that people are very reluctant to read content of any type that is not fiction or is of very high relevance to some particular niche interest of theirs. Inviting people to a workshop seems to work a lot more reliably to me, though obviously with written material you get a much broader reach, which can compensate for the lower conversion rate (and which medium makes sense to optimize I think hinges a lot on whether you care about getting a small specific set of people to learn something, vs. trying to get as many people as possible to learn something).

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-07T17:21:30.206Z · score: 4 (7 votes) · EA · GW
I disagree that 80k should transition towards a £3k retreat + no online content model, but it doesn't seem worth getting into why here.

I never said 80k should transition towards a retreat + no online content model. What I said is that it seems plausible to me it would still produce a lot of value in that case, though I agree that their current model seems likely a better fit for them, and probably overall more valuable. Presumably you also disagree with that, but it seemed important to distinguish.

"a previous statement or proposition from which another is inferred or follows as a conclusion"

Given that in the scenario as outlined, there was no "previous statement" or "previous proposition", I am still confused how you think this definition fits. In the scenario at hand, nobody first outlined their complete argument for why they think the claim discussed is true, and as such, there is no "previous statement or proposition" that can be referred back to. This definition seems to refer mostly to logical argument, which doesn't really apply to most human cognition.

I am not super excited about debating definitions, and we both agree that using the word premise is at least somewhat close to the right concept, so I am not very excited about continuing this thread further. If you really care about this, I would be glad to set up an experiment on mechanical turk in which we ask participants to list the necessary premises of a belief they hold, and see how much their responses differ from asking them what observations would change their mind about X. It seems clear to me that their responses would differ significantly.

which premises you disagree on cause your disagreement.

This is still only capturing half of it, even under the definition of premise that you've outlined here, which seems to be a reasonable definition of what a crux for a single participant in the conversation is. A double crux would be "a set of premises, that when viewed as a new conjunctive proposition you both assign opposite truth values to, that when flipped would cause both of you to change their mind". Though that alone obviously doesn't yet make a procedure, so there is still a bunch more structure, but I would think of the above as an accurate enough description to start working with it.

It is logically impossible that double cruxing extends this characterisation.

I don't think I really know how to engage with this. Obviously it's possible for double-crux to extend this characterization. I even outlined a key piece that was missing from it in the above paragraph.

But it's also a procedure that is meant to be used with real people, where every bit of framing and instruction matters. If you really believe this, let us run a test and just give one group of people the instruction "find the premises on which you disagree on that cause your disagreement" and the other group the full double crux worksheet. Presumably you agree that the behavior of those groups will drastically differ.

You maybe have something more specific in mind when you mean "logically impossible", but given that we are talking about a high-level procedure proofs of logical impossibility seem highly unlikely to me.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-07T16:50:10.270Z · score: 10 (5 votes) · EA · GW

I don't think most of the costs that I described that come from legibility differ that much between research and educational institutions? The american public education system, as well as many other public education systems actually strike me as core examples of systems that have suffered greatly due to very strong forces on legibility in all of their actions (like standardized curricula combined with standardized testing). I think standardized testing is pretty good in a lot of situations, but that in this case it resulted in a massive reduction in variance in a system where most of the value comes from the right tail.

I agree that there are also other separate costs to legibility in cutting-edge domains, but the costs on educational institutions still seem quite significant to me. And most of the costs are relatively domain-general.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-07T04:27:16.993Z · score: 21 (12 votes) · EA · GW
On the payment model, imagine that instead of putting their material on choosing a high impact career online, 80k charged people £3000 to have 4 day coaching and networking retreats in a large mansion, afterwards giving them access to the relevant written material.

CFAR's model is actually pretty similar to 80k's here. CFAR generally either heavily discounts or waives the cost of the workshop for people they think are likely to contribute to the long-term-future, or are more broadly promising, and who don't have the money to pay for the workshop. As such the relevant comparison is more "should 80k offer paid coaching (in addition to their free coaching) at relatively high rates for people who they think are less likely to contribute to improving the world, if the money they earn from that allows them to offer the other free coaching services (or scale them up by 30% or something like that)", to which my answer would be "yes".

My sense is that 80k is in a better-funded position, and so this tradeoff doesn't really come up, but I would be surprised if they never considered it in the past (though career coaching is probably somewhat harder to monetize than the kind of product CFAR is selling).

I also think you are underestimating to what degree the paid workshops were a necessity for CFAR historically having gotten to exist. Since there is a lot of downtime cost in being able to run workshops (you need to have a critical mass of teaching staff, you need to do a lot of curriculum development, have reliable venues, etc.) and the EA community didn't really exist yet when CFAR got started, it was never really an option for CFAR to fully run off of donations, and CFAR additionally wanted to make sure it actually produced something that people would be willing to pay for, so offering paid workshops was one of the only ways to achieve those two goals. I also generally think it's a good idea for projects like CFAR to ensure that they are producing a product that people are willing to pay significant amount of money for, which is at least a basic sanity check on whether you are doing anything real.

As an example, I encouraged Lynette to ask people whether they would be willing to pay for her coaching, and ideally ask them for at least some payment even if she can't break even, to make sure that the people she is offering services to are filtered for the people who get enough value out of it to spend $50 per session, or something in that space (she had also considered that already on her own, though I don't remember the current state of her asking her clients for payment).

I just remembered that 80k actually did consider monetizing part of it's coaching in 2014, which would have probably resulted in a pretty similar model to CFAR:

Is there a subsection of the audience who might be willing to pay for coaching?
We’re interesting in the possibility of making part of the coaching self-funding. Our best guess was that the people who will be most willing to pay for coaching are people from tech and finance backgrounds aged 25-35. We found that about 20% of the requests fell in this category, which was higher than our expectations.

Re retreats:

I think it's quite plausible that 80k organizing retreats would be quite valuable, in particular in a world where CFAR isn't filling that current niche. CEA also organized a large number of retreats of a similar type in the last year (I attended one on individual outreach, and I know that they organized multiple retreats for group organizers, and at least one operations retreat) presumably because they think that is indeed a good idea (the one that I attended did seem reasonably valuable, and a lot of the design of it was clearly influenced by CFAR workshops, though I can't speak on whether that overall initiative was worth it).

afterwards giving them access to the relevant written material

I agree that 80k also has a lot of impact via their written material, but I think that is because they have invested a very large fraction of their resources into producing those materials (80K would likely be unable to run as many workshops as CFAR and also produce the written material). I think if 80k was focusing primarily on coaching, it would be very unlikely to produce good written material that would stand well on its own, though I expect it would still produce a good amount of value (and it might still produce some writings, but likely not ones that make much sense without the context of the coaching, similar to CFAR). As such I am skeptical of your claim that switching to that model would get rid of ~100% of 80k's value. I expect it would change their value proposition, but likely still have a good chance of being competitive in terms of impact (and fully switching towards a coaching model was something that I've heard 80k consider multiple times over the years).

Your answer could be expressed in the form of premises right? Premises are just propositions that bear on the likelihood of the conclusion

I think if you define "premise" more broadly to mean "propositions that bear on the likelihood of the conclusion" then you are closer, but still not fully there. A crux would then be defined "a set of premises that when falsified, would provide enough evidence that you would change your mind on the high-level claim", which is importantly still different from "identifying differences in our premises", in particular it emphasizes identifying specific premises that are particularly load-bearing for the argument at hand.

(This wouldn't be a very standard usage of "premise" and doesn't seem to align super well with any definitions I can find in any dictionaries, which all tend to either be about logical inference or about subsets of a specific logical argument that is being outlined, but doesn't seem like a horrible stretch from available definitions. Though I wouldn't expect people to intuitively know what you mean by that definition of "premise")

I do still expect people to give quite drastically different answers if you ask them "is 'not X' a premise of your belief?" vs. "would observing X change your mind about this belief?". So I wouldn't recommend using that definition if you were actually trying to do the thing that double crux is trying to do, even if you define it beforehand. I do think that the norms from (classical) rhetoric and philosophy of trying to identify differences in your premises are good norms and generally make conversations go better. I agree that Double Crux is trying to operationalize and build on that, and isn't doing some weird completely novel thing, though I do think it extends on it in a bunch of non-trivial ways.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-07T04:26:59.491Z · score: 16 (11 votes) · EA · GW

To give an answer to the question of what material that CFAR teaches at their workshops I consider valuable, here is a list of classes that I've seen have a big impact on individuals, sometimes including myself and for which I also have separate reasons to think they are valuable.

  • Units of Exchange
    • Basically an introduction into consequentialist reasoning, trying to get people to feel comfortable trading off different resources that they previously felt were incomparable. A lot of the core ideas in EA are based off of this, and I think it's generally a good introduction into that kind of thinking.
  • Inner Simulator
    • Basic practical introduction into System 1 and System 2 level processing, and going into detail on how to interface between S1 and S2 processing.
  • Trigger-Action Planning
    • Basic introduction into how associative processing works in the brain, where it tends to work, and where it tends to fail, and how to work around those failure modes. In the literature the specific technique is known as "Mental contrasting with implementation intentions" and is probably one of the most robust findings in terms of behavior change in behavioral psychology.
    • This class is often particularly valuable because I've seen it provide people with their first real mechanistic model of the human mind, even if simplified. A lot of people don't really have any mechanistic baseline of how human cognition works, and so the simplified statement of "humans cognition can be modeled as a large pile of programmed 'if-then-statements' can get people initial traction on figuring out how their own mind works".
  • Goal Factoring
    • For most attendees this has a lot of overlap with basic 80k coaching. Practice in trying to ask yourself repeatedly "why is this thing that I am doing important to me, and could I achieve it some better way?", and this is probably the class that I've seen that had the biggest effects in terms of causing career changes in participants, mostly by getting them to think about why their are pursuing the career they are pursuing, and how they might be able to achieve their goals better.
  • Understanding Shoulds
    • This covers a lot of material in the Minding Our Way "Replacing Guilt" series, which many EAs and people that I trust have reported to have benefited a lot from, and which core conclusions are quite important for a lot of thinking about how to have a big impact in the world, how morality works, reminding people that they are allowed to care about things, etc.
  • Focusing
    • Based on Gendlin's "Focusing" book and audiobook, it teaches a technique that forms the basis of a significant fraction of modern therapeutic techniques and I consider a core skill for doing emotional processing. I've benefited a lot from this, and it also has a pretty significant amount of evidence behind it (both in that it's pretty widely practiced, and in terms of studies), though only for the standards of behavioral psychology, so I would still take that with a grain of salt.
  • Systemization
    • This is basically "Getting Things Done" the book, in a class. I, and a really large number of people I've worked with and who seem to be good at their job, consider this book core reading for basically anyone's personal productivity, and I think teaching this is pretty valuable. This class in particular tends to help people who bounced off of the book, which still recommends a really large fraction of practices that I've seen in particular young people bounce off of, like putting everything into binders and getting lots of cabinets to put those binders in, instead of having good digital systems.
  • Double Crux
    • We've discussed this one above a good amount. In particular I've seen this class cause a bunch of people to have productive conversations that have previously had dozens of hours of unproductive or really conflict-heavy conversations, the most easily referenced and notable of which is probably a conversation between Scott Garrabrant and Eric Drexler that I think significantly moves the conversation around AI Alignment forward

All of the above strike me as pretty robustly good concepts to teach, already make up more than 50% of intro workshops, and that are pretty hard to get a good grasp on without reading ~6 books, and having substantial scaffolding to actually put time into practicing the relevant ideas and techniques.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-06T22:29:15.944Z · score: 8 (2 votes) · EA · GW

It's a bit of a more playful term, which I think makes sense in the context of a workshop, but I also use the two terms interchangeably and seen CFAR staff do the same, and usually use pre-mortem when I am not in a CFAR context.

I don't have strong opinions on which term is better.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-06T18:49:54.152Z · score: 10 (13 votes) · EA · GW

Hmm, it seems to me like you are modeling the goals and purpose of CFAR quite differently than I do. I model CFAR primarily as an educational institution, with a bit of research, but mostly with the goal of adapting existing knowledge and ideas from cognitive science and other disciplines into more practical applications (hence the name "Center for Applied Rationality").

In my review of CFAR last round, I listed

  • "Establishing Epistemic Norms"
  • "Recruitment" and
  • "Training"

as the three primary sources of value add of CFAR, which importantly lacks what you seem to be evaluating above, and I would describe as "research" (the development of new core concepts in a given field).

I think in the space of the three axes I outlined, CFAR has been pretty successful, as I tried to explain during my last review, and with some additional evidence that I have heard particularly good things about the Artificial Intelligence Risk for Computer Scientist workshops, in that they seem to be able to facilitate a very unique environment in which people with a strong technical background can start engaging with AI Alignment questions.

I don't think of CFAR's value being primarily generated by producing specific insights of the type of Kahneman's work (though I do think there have been some generated by CFAR that I found useful) but in the teaching and communication of ideas in this space, and the feedback loop that comes from seeing how trying to teach those techniques actually works (often uncovering many underspecified assumptions, or the complete lack of effectiveness of an existing technique).

Murphyjitsu is indeed just pre-mortem, and I think is cited as such in both the handbook and at the workshop. It's just that the name of premortem didn't stick with participants, and so people changed it to something that people seemed to actually be able to engage with (and there was also a bunch of other valuable iteration on how you actually teach the relevant skill).

I also wonder why CFAR has to charge people for their advice. Why don't they write down all of their insights and put it online for free?

This seems to again approach CFAR's value add from a different perspective. While I would be in favor of CFAR publishing their handbook, it's clear to me that this would not in any real way compete with the value of existing CFAR workshops. Universities have classes, and very few people are able to learn from textbooks alone, and their and CFAR's value comes from the facilitation of classes, active exercises, and the fast feedback loop that comes from having an instructor right in the room with you.

I am in favor of CFAR writing more things down, but similar to how very few people can learn calculus or linear algebra from nothing but a book, with no accountability structure or teacher to ask questions off, is it also unlikely that many people can learn the relevant subsets of cognitive science and decision-making from just a written description (I think some can, and our community has a much higher fraction of autodidacts than the general population, but even for those, learning without a teacher is usually still a lot slower).

I do think there are other benefits to writing things down, like having more cross-examination of your ideas by others, giving you more information about them, "just writing down their ideas" would not effectively replace CFAR's value proposition.

-----

I am however just straightforwardly confused by what you mean by "isn't double crux merely agreeing on which premise you disagree on?", since that seems to have relatively little to do with basically any formulation of double crux I've seen. The goal of double crux is to formulate which observations would cause both you and the other person to change their mind. This has at most a tenuous connection to "which premise do we disagree on?", since not all premises are necessary premises for a conclusion, and observations tend to only very rarely directly correspond to falsifying one specific premise. And human cognition usually isn't structured by making explicit premises and arguing from them, making whatever methodology you are comparing it to not really be something that I have any idea how to apply in conversation (if you ask me "what are my premises for the belief that Nature is the most prestigious science journal?" then I definitely won't have a nice list of premises I can respond with, but if you ask me "what would change my mind about Nature being the most prestigious science journal?" I might be able to give a reasonably good answer and start having a productive conversation).

Comment by habryka on [Link] What opinions do you hold that you would be reluctant to express in front of a group of effective altruists? Anonymous form. · 2019-10-06T04:48:27.033Z · score: 5 (3 votes) · EA · GW

*nods* seems good.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-05T16:56:15.538Z · score: 5 (3 votes) · EA · GW

That's good to hear! Because a lot of what I described above was a relatively informal procedure, I felt weird putting a lot of emphasis on it in the writeup, but I do agree that it seems like important information for others to have.

I think by next round we will probably have a more formal policy that I would feel more comfortable explicitly emphasizing in the writeup.

Comment by habryka on [Link] What opinions do you hold that you would be reluctant to express in front of a group of effective altruists? Anonymous form. · 2019-10-05T16:53:12.586Z · score: 5 (3 votes) · EA · GW

I think an anonymous poll of that type is probably fine, though just asking for offensive ideas is probably less likely to get valuable responses than the OP, so I feel less strongly about people being able to make that type of poll happen.

I do however still think that knowing the answers to that poll would be reasonably useful, and I still expect this to help me and others build better models of what others believe, and also think there is a good chance that a poll like this can break an equilibrium in which a silent majority is unwilling to speak up, which I think happens quite a bit and is usually bad.

So yeah, I think it would be fine to organize that poll. It's a bit of a weird filter, so I would have some preference for the person adding an explicit disclaimer that this is an anonymous internet poll and ultimately this is primarily a tool for hypothesis generation, not a representative survey, but with that it seems likely reasonably positive to me. I don't feel like that survey is as important as the type of survey that the OP organized, but I wouldn't want to punish a person for organizing it, or filling it out.

Comment by habryka on [Link] What opinions do you hold that you would be reluctant to express in front of a group of effective altruists? Anonymous form. · 2019-10-05T05:08:11.629Z · score: 24 (11 votes) · EA · GW

I do indeed generally think that whether their writings will "damage the movement" should not be particularly high in their list of considerations to think about when asking other people questions, or writing up their thoughts. I think being overly concerned with reputation has a long history of squashing intellectual generativity, and I very explicitly would not want people to feel like they have to think about how every sentence of theirs might reflect on the movement from the perspective of an uncharitable observer.

I prefer people first thinking about all the following type of considerations, and if the stakes seem high-enough, maybe also add reputation concerns, though the vast majority of time the author in question shouldn't get that far down the list (and I also note that you are advocating for a policy that is in direct conflict with at least one item on this list, which I consider to be much more important than short-term reputation concerns):

  • Are you personally actually interested in the point you are making or the question you are asking?
  • Does the answer to the question you are asking, or answering, likely matter a lot in the big picture?
  • Is the thing that you are saying true?
  • Are you being personally honest about your behavior and actions?
  • Are you making it easier for other people to model you and to accurately predict your behavior in the future?
  • Does your question or answer address a felt need that you yourself, or someone you closely interacted, with actually has?
  • Are you propagating any actually dangerous technological insights, or other information hazards?

I would strongly object to the norm "before you post to the forum, think very hard about whether this will damage the reputation of the movement", which I am quite confident would ensure that very little of interest would be said on this forum, since almost all interesting ideas that have come out of EA are quite controversial to many people, and also tended to have started out in their least polished and most-repugnant form.

I also remember the closing talk of EAG 2017, with the theme being "stay weird", that explicitly advocated for being open and welcoming to people who say things that might sound strange or unpopular. I think that reflected an understanding that it is essential for EA to be very welcoming of ideas that sound off putting and heretical at first, in particular if they are otherwise likely to be punished or disincentivized by most of society.

From a blogpost by Scott Alexander:

But I got a chance to talk to [Will MacAskill] – just for a few minutes, before he had to run off and achieve something – and I was shocked at how much he knew about all the weirdest aspects of the community, and how protective he felt of them. And in his closing speech, he urged the attendees to “keep EA weird”, giving examples of times when seemingly bizarre ideas won out and became accepted by the mainstream.

I think a key example in this space would be a lot of the work by Brian Tomasik, whose writing I think is highly repugnant to large fractions of society, but has strongly influenced me in my thinking, and is what I consider to be one of the most valuable bodies of work to come out of the community (and to embody its core spirit, of taking ethical ideas seriously and seeing where they lead you), even though I strongly disagree with him on almost every one of his conclusions.

So no, I don't think this is a good norm, and would strongly advise against elevating that consideration to the short list of things that people actually have the mental energy for to do when posting here. Maybe when you are writing an article about EA in a major newspaper, but definitely not for this forum, the most private space for public discourse that we have, and the primary space in which we can evaluate and engage with ideas in their early stages.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-05T04:49:02.957Z · score: 4 (2 votes) · EA · GW

I would definitely also be interested in talking about this, either somewhere on the forum, or in private, maybe with a transcript or summarized takeaways from the conversation posted back to the forum.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-05T04:48:00.960Z · score: 16 (7 votes) · EA · GW

Ok, let me go into more detail on that.

I think the biggest obstacle with funding initiatives like this is definitely that it's very hard to even just identify a single potentially promising project without looking into the space for quite some time. We don't really have resources available for extensive proactive investigation into grant areas, so someone I reasonably trust suggesting this as a potential meta-science initiative is definitely the biggest reason for us making this grant.

In general, as I mentioned in one of the sections in the writeup above, we are currently constrained to primarily do reactive grantmaking, and so are unlikely to fund projects that did not apply to the fund or were high on our list of obvious places to maybe give money to.

I have a strong interest in meta-science initiatives, and Chris Chambers was the only project this round that applied in that space, so that combination was definitely a major factor.

However, I do also think that Chambers has achieved some pretty impressive results with his work so far:

Chambers keeps an online spreadsheet with all the journals that have adopted the format [262].
To date, 140 journals have adopted them so far and the fields covered are:
+ Life/medical sciences: neuroscience, nutrition, psychology, psychiatry, biology, cancer research, ecology, clinical & preclinical medicine, endocrinology, agricultural and soil sciences
+ Social sciences: political science, financial and accounting research
+ Physical sciences: chemistry, physics, computer science etc.
+ Generalist journals that cover multiple fields: Royal Society Open Science and Nature Human Behaviour

His success so far has made this one of the most successful preregistration projects I know of to-date, and it seems likely that further funding will relatively straightforwardly generalize to more journals offering registered-reports as a potential way to publish.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-05T04:36:38.056Z · score: 5 (3 votes) · EA · GW

Thanks!

I agree, and am also excited to see updates on their work. I've updated on the importance of follow-up investigations since last round, so I might invest less resources in the writeups next round, and invest some additional resources into following up with past grantees and getting a sense of how their projects played out.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-05T04:35:22.336Z · score: 16 (4 votes) · EA · GW

Thank you! I agree with this assessment. My current guess is that it doesn't necessarily make sense for everyone to run my strategy of openness and engaging in lots of discussion, but that at the margin I feel like I would like to see a lot more of that.

I also have the same sense of feeling like Bridgewater culture is both a good example and something that illustrates the problems of doing this universally.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-05T04:27:10.671Z · score: 9 (5 votes) · EA · GW

I like this metric. I agree that it would be quite challenging to meet, but would definitely be a decent indicator of at least reach and readership. Obviously I wouldn't want it to be the only metric, but it seems like a good one to look into for any project like this.

Yeah, I do think I could do a bit better at defining what I would consider success for this grant, so I will try to write a comment with some more of my thoughts on that in the next few days.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-04T05:25:15.078Z · score: 34 (16 votes) · EA · GW

Happy to describe what we've historically done, though we are still figuring out some better policies here, so I expect this to have changed by next round.

As I mentioned in the description of our voting process last time, we have a spreadsheet that has all the applications and other organizations we are considering recommending grants to, in which each fund member can indicate a vote as well as a suggested funding amount (we are still iterating on whether to use votes or a more budget-based system). Whenever there is a potential cause for a conflict of interest, the relevant person (or a separate fund member who suspects a COI) leaves a comment in the relevant row in the spreadsheet with the details of the COI, then the other fund members look at the description of the COI and decide whether it makes sense for that person to withdraw from voting (so far the fund always agreed with the assessment of the individual fund member on whether they should withdraw, but since any fund member has veto power over any of our grant recommendations, I am confident that we would not make grants in which one fund member thinks that a different fund member has a strong COI and we don't have independent evidence supporting the grant).

We don't (yet) have a super concrete policy of what counts as a conflict of interest, but I think we've historically been quite conservative and flagged all the following things as potential conflicts of interest (this doesn't mean I am certain that all of the below have been flagged on every occasion, though I think that's quite likely, just that all of these have historically been flagged) :

  • Having worked with the relevant organization or people in the past (this usually just gets flagged internally and then added to the writeup, and doesn't cause someone to withdraw from voting)
  • Living in the same house/apartment as the person we are granting to (usually makes us hesitant to make a grant just on the basis of the relevant person, so we usually seek additional external feedback to compensate for that)
  • Being long-time friends with the potential grantee (I expect would just get flagged and added to the writeup)
  • Being a past or current romantic partner of the potential grantee (I expect this would cause someone to exclude themselves from voting, though I don't think this ever became relevant. There is one case where a fund member first met and started dating a potential grantee after all the votes had been finalized, but I don't think there was any undue influence in that case.)
  • Having some other interpersonal conflict with the relevant person (This usually doesn't make it into the writeup, but I flagged it on one occasion)
  • Probably some others, since COIs can arise from all kinds of things

If the votes of the fund member with the potential COI mattered in our final grant decision we've incorporated descriptions of those COIs into the final writeups, and sometimes added writeups by people who have less cause for a COI to provide a more neutral source of input (an example of this is the Stag grant, which has a writeup by Alex who has some degree of COI in his relationship to Stag, so it seemed good to add a writeup by me as an additional assessment of the grant).

CEA is currently drafting a more formal policy which is stricter about fund members making recommendations to their own organizations, or organizations closely related to their own organization, but doesn't cover most of the other things above. We are also currently discussing some more formalized COI policy internally, though I always expect that for the vast majority of potential COI causes we will have to rely on a relatively fuzzy definition, because these can arise from all kinds of different things.

Comment by habryka on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-03T22:10:27.671Z · score: 7 (8 votes) · EA · GW

Of the categories listed, that seems to most accurately summarize what happened (edit: though I don't think this should be seen as a concrete endorsement of CFAR's long-term plans, and have more to do with the considerations about decision-making ability I outlined above). I do think that it's still quite possible that in future rounds I will take the first option, and make recommendations conditional on some changes, though I feel comfortable with the amount of funding we recommended to CFAR this round.

CFAR also communicated to me that they plan to focus more on external transparency in the coming months, so I expect that I will continue building out my models in this space.

Comment by habryka on EA Meta Fund and Long-Term Future Fund are looking for applications again until October 11th · 2019-10-03T18:49:59.217Z · score: 7 (4 votes) · EA · GW

It's up!

Comment by habryka on [Link] What opinions do you hold that you would be reluctant to express in front of a group of effective altruists? Anonymous form. · 2019-10-03T18:35:46.904Z · score: 5 (7 votes) · EA · GW

It is unclear whether the author has put thought into the downsides, all we know is that the author did not emphasize potential downsides in the writeup.

I don't think the person doesn't have to put any thought into whether publishing a post like this is a good idea or not, only that they don't have to put significant effort into publicly making a case for the benefits outweighing the cost. The burden of making that case is much larger than the burden of just thinking about it, and would be large enough to get rid of most people just asking honest questions of others in public.

Comment by habryka on [Link] What opinions do you hold that you would be reluctant to express in front of a group of effective altruists? Anonymous form. · 2019-10-03T07:32:15.025Z · score: 8 (14 votes) · EA · GW

I do think that I am quite hesitant to promote a norm that you are no longer allowed to ask people questions about their honest opinion in public, without having written a whole essay about the possible reasons for why that might be bad. I don't think this is the type of question that one should have to justify, it's the type of question that our community should make as easy as possible.

There exist risks, of course, but I think those risks should be analyzed by core members of the community and then communicated via norms and social expectations. I don't think it's reasonable to expect every member of the community to fully justify actions like this.

Comment by habryka on [Link] What opinions do you hold that you would be reluctant to express in front of a group of effective altruists? Anonymous form. · 2019-10-03T03:29:24.375Z · score: 19 (16 votes) · EA · GW

Just as a sign of social support: I am grateful to whoever organized this poll, and would be deeply saddened to be part of a community where we punish people who organize polls like this. Obviously it's fine for Halstead to have his perspective, but it seemed valuable to provide a counterpoint to communicate that I would be happy to defend anyone who organizes polls like this, and put a significant fraction of my social capital behind our collective ability to do things like this.

Comment by habryka on EA Meta Fund and Long-Term Future Fund are looking for applications again until October 11th · 2019-10-03T02:33:15.490Z · score: 2 (1 votes) · EA · GW

Alas, it's been delayed by a bit, but it should go up this week. Sorry for the confusion!

Comment by habryka on Survival and Flourishing Fund Applications closing in 3 days · 2019-10-03T02:32:46.840Z · score: 3 (2 votes) · EA · GW

My guess is in around 4 months, though I don't know of any concrete plans. So take this as a highly tentative prediction.

Comment by habryka on [Link] What opinions do you hold that you would be reluctant to express in front of a group of effective altruists? Anonymous form. · 2019-09-27T05:07:51.035Z · score: 43 (22 votes) · EA · GW

One of the form responses said:

Long-term future fund literally just gave their friends money for no reason / because one of the decision makers was hooking up with an applicant.

As a person who cares a lot about the long-term future fund, I wanted to mention that I am unaware of anything like this happening (I don't think any grants we've made have been to anyone who has ever been romantically involved with any of the fund members). If the person has additional evidence for this, I am interested in hearing it, and would offer full confidentiality and promise to investigate this extensively if sent to me. I think it is quite important for any member of the Long-Term-Future Fund to disclose any conflicts of interests of this kind, and would take any violation of that very seriously.

Obviously if the person who wrote this seems uncomfortable reaching out to me for any reason, reaching out to any of the trustees of CEA, or Julia Wise is likely also an option, who I also expect to be open to commit to full confidentiality on this.