Posts
Comments
Thanks, that was useful. I didn't realise that his argument involved 1+2 and not just 1 by itself. That said, if the hinge of history was some point in the past, then that doesn't affect our decisions as we can't invest in the past. And perhaps it's a less extraordinary coincidence that the forward-looking hinge of history (where we restrict the time period from now until the end of humanity) could be now, especially if in the average case we don't expect history to go on much longer.
I've never found Will's objections to the hinge of history argument persuasive. Convincing me that there was a greater potential impact in past times than I thought, ie. that it would have been very influential to prevent the rise of Christianity, shouldn't make me disbelieve that arguments that AI or bio risks are likely to lead to catastrophe in the next few decades if we don't do anything about it. But maybe I just need to reread the argument again.
DNA engineering has some positive points, but imagine the power that having significant control its citizens personalities would give the government. That shouldn't be underestimated.
The real hinge here is how much we should expect the future to be a continuation of the past and how much we update based on our best predictions. Given what we know about about existential risk and the likelihood that AI will dramatically change our economy, I don't think this idea makes sense in the current context.
I agree that such a system would be terrifying. But I worry that its absence would be even more terrifying. Limited surveillance systems work decently for gun control, but when we get to the stage where someone can kill tens of thousands or even millions instead of a hundred I suspect it'll break down.
Thanks for posting such a detailed answer!
It's great to hear that you are setting this up. However, the current post seems light on details. Why are these areas of particular interest? What kind of commitment are you hoping for from participants?
I think people are quite reasonably deciding that this post isn't worth taking the time to engage with. I'll just make three points even though I could make more:
"A good rule of thumb might be that when InfoWars takes your side, you probably ought to do some self-reflection on whether the path your community is on is the path to a better world." - Reversed Stupidity is Not Intelligence
"In response, the Slate Star Codex community basically proceeded to harass and threaten to dox both the editor and journalist writing the article. Multiple individuals threatened to release their addresses, or explicitly threatened them with violence." - The author is completely ignoring the fact that Scott Alexander specifically told people to be nice, not to take it out on them and didn't name the journalist. This seems to suggest that the author isn't even trying to be fair.
"I have nothing to say to you — other people have demonstrated this point more clearly elsewhere" - I'm not going to claim that such differences exist, but if the author isn't open to dialog on one claim, it's reasonable to infer that they mightn't be open to dialog on other claims even if they are completely unrelated.
Quite simply this is a low quality post and "I'm going to write a low quality post on topic X and you have to engage with me because topic X is important regardless of the quality" just gives a free pass on low quality content. But doesn't it spur discussion? I've actually found that most often low quality posts don't even provide the claimed benefit. They don't change people's minds and tend to lead to low quality discussion.
It's worth remembering though that when people who paid for the book are much more likely to have read it
What do you think about the fact that many in the field are pretty open that they are pursuing enquiry on how to achieve an ideology and not neutral enquiry (using lines like all fields are ideological whether they know it or not)?
"It's a bit concerning that the community level of knowledge of the bodies of work that deal with these issues is just average" - I do think there are valuable lessons to be drawn from the literature, unfortunately a) lots of the work is low quality or under-evidenced b) discussion of these issues often ends up being highly divisive, whilst not changing many people's minds
"If the self-oriented reasons for action leave it largely underdetermined how personal flourishing would look like" - If we accept pleasure and pain, then we can evaluate other actions in high likely they are to lead to pleasure/pain in the long term, so I don't see how actions are underdetermined.
I'm surprised that you put moral realism on the same tier as self-oriented reasons for action. It would seem much more astounding to claim that pain and pleasure are neither good nor bad *for me*, then to claim that there's no objective stance by which others should consider my pain good or bad. The Pascal's wager argument is also much stronger too.
I think percentages are misleading. In terms of influencing demographic X, what matters isn't so much how many people of demographic X there are in these organisations, but how well-respected they are.
I'm generally against evaluating diversity programs by how much diversity they create. It's definitely a relevant metric, but we don't evaluate AMF by how many bednets they hand out, but the impact of these bednets.
I guess that seems so far off that I wasn't focusing on it. I'm more interesting in how to establish a working impact purchase in the mean time.
"These people where encouraging me to use my last savings to retrain to a risky career, but putting in their money was out of the question" - Yeah, I'm sorry you had that experience, that seems unpleasant. Maybe they didn't understand the financial precariousness of your situation? Like many EAs wouldn't find it hard to land a cushy job in normal times and likely have parents who'd bail them out worst comes to worst and might have assumed that you were in the same position without realising they were making an assumption?
Or if you think an outcome was mostly bad luck, fund them more than just impact purchase
Yeah, luck is another argument I considered covering, but didn't get into. Sometimes the impact of a project is just a matter of being at the right place in the right time. Of course, it's hard to tell; to a certain extent people make their own luck.
But in most cases I would suggest straight up impact purchase, because anything else is is really hard and you'll probably get the adjustments wrong.
I guess this would be a key point where we differ. I haven't thought deeply about this, but my intuition would be that adjustments would greatly improve impact. For example, a small project extremely competently implemented and a big project poorly implemented might have the exact same impact, but the former would be a stronger signal.
I wrote up my thoughts in this post: Making Impact Purchases Viable. Briefly, I argue that:
- Restrictions on sellers are necessary to fix and imbalance between the size of the buyer's market and seller's market
- Restrictions are also important for providing sufficient incentive for a reasonable proportion of the impact to be counterfactual
- Another issue I don't have a solution to is the potential of impact purchases to lead to bitterness or demotivate people
That line of reasoning also suggests that EA orgs should ask each of their employees whether they will work for free if they don't get a salary; and refuse to pay a salary to employees who answer "yes".
Maybe, but I imagine that the number of people who'd work just as hard long-term would be about zero, so more of the impact would be counterfactual.
Even though fair trade is ineffective on an individual level, it may be effective on a collective level because enough people find it appealing for broad adoption. Deciding to ignore it weakens any attempt to establish buying fairtrade as a society.
EAs don't arise out of a vacuum, but out of society. If society is doing well, then EAs are more likely to do well too and hence to have more impact. So by not donating to a local charity, you are refusing to invest in the society that provided you the chance to have an impact in the first place.
Not saying you should donate locally or buy fair trade, just pointing out one worry with ignoring them.
Thanks so much for writing this. I've had similar worries regarding local charity and things like fair trade for a while.
It's hard to do a summary without encouraging people to read the summary instead of the article.
My understanding from looking briefly was that Drawdown focused on total reduction potential, not cost-effectiveness
These diagrams look really useful for encouraging people to map out potential paths to existential risk and potential interventions more carefully!
I suspect that part of the reason why this is happened is that the EA community is closely associated with the rationality community, so it's often easiest just to have the personal development discussion online over there. Plus another reason that people mightn't feel a need for it online is that lots of discussion of personal development occurs informally at meetups.
What's Good Growth?
Yeah, EA is likely less compelling when this is defined as feeling motivating/interesting to the average person at the moment, although it is hard to judge since EA hasn't been around for anywhere near as long. Nonetheless, many of the issues EAs care about seem way too weird for the average person, then again if you look at feminism, a lot of the ideas were only ever present in an overly academic form. Part of the reason why they are so influential now is that they have filtered down into the general population in a simpler form (such as "girl power", "feeling good, rationality bad"). Plus social justice is more likely to benefit the people supporting it in the here and now than EA which focuses more on other countries, other species and other times which is always a tough sell.
SJ is an extremely inclusive movement (basically by definition)
I'm generally wary of argument by definition. Indeed, SJ is very inclusive to members of a racial minority or those who are LGBTI, but is very much not when it comes to ideological diversity. And some strands can be very unwelcoming to members of majorities. So it's much more complex than that.
"There are definitely many who see these more in the movement/tribe sense" - For modern social justice this tends to focus on who is a good or bad person, while for EA this tends to focus more on who to trust. (There's a less dominant strand of thought within social justice that says we shouldn't blame individuals for systematic issues, but it's relatively rare). EA makes some efforts towards being anti-tribal, while social justice is less worried about the downsides of being tribal.
Greater knowledge of psychology would be powerful, but why should we expect the sign to be positive, instead of say making the world worse by improving propaganda and marketing?
Why is Leverage working on psychology? What is it hoping to accomplish?
This seems like a good idea and definitely the thing I'd consider once I learn enough about ai that this would be valuable for others.
"It’s not clear that advanced artificial intelligence is going to arrive any time within the next several decades" - On the other hand, it's seems, at least to me, most likely that it will. Even if several more breakthroughs would be required to reach general intelligence, those may still come relatively fast as deep learning has now finally become useful enough in a wide enough array of applications that there is far more money and talent in the field than there ever was before by orders of magnitude. Now this by itself wouldn't necessarily guarantee fast advancement in a field, but AI research is still the kind of area where a single individual can push the research forward significantly just by themselves. And governments are beginning to realise the strategic importance of AI, so even more resources are flooding the field.
"One of the top AI safety organizations, MIRI, has now gone private so now we can’t even inspect whether they are doing useful work." - this is not an unreasonable choice and we have their past record to go on. Nonetheless, there are more open options if this is important to you.
"Productive AI safety research work is inaccessible to over 99.9% of the population, making this advice almost useless to nearly everyone reading the article." - Not necessarily. Even if becoming good enough to be a researcher is very hard, it probably isn't nearly as hard to become good enough at a particular area to help mentor other people.
I'm definitely in favour of this kind of project since I feel more EAs should be experimenting with small projects.
"The situation seems pretty symmetric, though: if a politician builds roads just to get votes, and an NGO steps in and does something valuable with that, the politician's counterfactual impact is still the same as the NGO's" - true, but the NGO's counterfactual impact is reduced when I feel it's fairer for the NGO to be able to claim the full amount (though of course you'd never know the government's true motivations in real life)
The order indifference of Shapely values only makes sense from a perspective where there is perfect knowledge of what other players will do, but if you don't have that, a party that spent a huge amount of money on a project that was almost certainly going to be wasteful and ended up being saved when by sheer happenstance another party appeared to save the project was not making good spending decisions. Similarly, many agents won't be optimising for Shapely value, say a government which spends money on infrastructure not caring about whether it'll be used or not just to win political points, so they don't properly deserve a share of the gains when someone else intervenes with notifications to make the project actually effective.
I feel that this article presents Shapley value as just plain superior, when instead a combination of both Shapley value and counterfactual value will likely be a better metric. Beyond this, what you really want to use is something more like FDT where you take into account the fact that the decisions of some agents are subjunctively linked to you and that the decisions of some other agents aren't. Even though my current theory is that very, very few agents are actually subjunctively linked to you, I suspect that thinking about problems in this fashion is likely to work reasonably well in practise (I would need to dedicate a solid couple of hours in order to be able to write out my reasons for believing this more concretely)
If we run any more anonymous surveys, we should encourage people to pause and consider whether they are contributing productively or just venting. I'd still be in favour of sharing all the responses, but I have enough faith in my fellow EAs to believe that some would take this to heart.
I'm most concerned about attempts to politicise the movement as unlike most of the other risks, this risk is adversarial. EA has to thread the needle of operating and maintaining our reputation in a politicised environment without letting this distort our way of thinking.
I suspect that it could be impactful to study say a masters of AI or computer science even if you don't really need it. University provides one of the best opportunities to meet and deeply connect with people in a particular field and I'd be surprised if you couldn't persuade at least a couple of people of the importance of AI safety without really trying. On the other hand, if you went in with the intention of networking as much as possible, I think you could have much more success.
Interesting reading your strategy, particularly what you aren't focusing on. The one part I'd be somewhat skeptical of is decreasing upskilling. People, particularly the people that we want to join our community, want to grow and improve. It's important to be realistic about how much someone can upskill in a limited amount of time, but these kinds of events seem like a key draw.
One of the vague ideas spinning around in my head is that maybe in addition to EA which is a fairly open, loosely co-ordinated, big-tent movement with several different cause areas, there would also in value in a more selective, tightly co-ordinated, narrow movement focusing just on the long term future. Interestingly, this would be an accurate description of some EA orgs, with the key difference being that these orgs tend to rely on paid staff rather than volunteers. I don't have a solid idea of how this would work, but just thought I'd put this out there...
That is pretty concerning. I would love an explanation of this as well!
I'm strongly in favour of creating a fellowship with a fancy name and website in order to allow people to build career capital; or at least make accepting these fellowships not a step backwards. EA Grant doesn't exactly sound prestigious.
"I have seen EAs circulate half-baked fears of suicide drones making it easy to murder people (just carry a tennis racket, or allow municipalities to track or ban drone flights if they so choose) or assassinate political leaders (they already speak behind bullet-resistant plexiglass sometimes, this is not a problem)" - Really not that easy. A tennis racket? Not like banning drones stops someone flying a drone from somewhere else. And political leaders sure you can speak behind the glass, but are you going to spend your whole life behind a screen?
Maybe EA should grow more, but I don't think that the issue is that we are "not ruthless enough". Instead I'd argue that meta is currently undervalued, at least in terms of donations.
People often assume that tagging is strictly better than sub-forums because it is more flexible, but categories have advantages too. For one, it is easier to filter them out, since there are less categories. Additionally, if you visit one category, then another, you are less likely to see duplicate posts.
Wow, they all sound so fascinating!
"What if you share short-termists’ skepticism of weird claims and hypothetical risks, but you’re willing to focus on first-principles reasoning and work on a long time scale?" - then you'd probably focus on nuclear which isn't at all hypothetical
Here's the link: https://meaningness.com/geeks-mops-sociopaths
Sorry, I can't respond to this in detail, because the conversation was a while back. Further, I don't have independent confirmation on any of the factual claims.
I could PM you one name they mentioned for point three, but out of respect for their privacy I don't want to post this publicly. Regarding point four, they mentioned article as a description of the dynamic they were worried about.
In terms of resources being directed to something that is not the mission, I can't remember what was said by these particular people, but I can list the complaints I've heard in general: circling, felon voting rights, the dispute over meat at EAG, copies of HPMoR. Since this is quite a wide spread of topics, this probably doesn't help at all.
"EAs seem to mostly interact with research groups and non-profits" - They were talking more about the kinds of people who are joining effective altruism than the groups we interact with