EA Forum Reader

The HN thread was definitely interesting, since the pivot of Open AI to a limited for profit company occurred, I am certainly concerned about whether they're going to be allocating as many resources for safety as they were and will be much more focused on commercial application development (which I think is a fair, and probably correct, thing to do when you have shareholders who have invested in you for a potential return rather than as a donor to a specific mission)

A note - you can easily find Greg Brockman (the Cofounder and CTO of Open AI) in the thread by his username gdb. One of the more interesting things that he mentions is that they may keep more tech private or for commercial use, and gives a very soft maybe on being able to possibly eventually publish it.

Whoops, sorry about the quotes--I was writing quickly and intended them to denote that I was using "solve" in an imprecise way, not attributing the word to you, but that is obviously not how it reads. Edited.

Nice work, again!

In future years I'd be very interested in seeing a citizenship question and a question about what city someone was in when they first became involved in EA. In Australia it's a running joke that we have an EA brain drain to USA/UK, especially for direct work opportunities (noting also that Australia has many fewer direct work opportunities as is also a way to read the data).

Finding opportunities for people to do good without needing to relocate (sometimes leaving a vacuum in community building in the city they are from) is something we're very interested in exploring further.

Yes, I agree they're very incomplete--as advertised. I also think the original claims they're responding to are pretty incomplete.

I.

I agree that time horizons are finite. If you're taking that as meaning that the defect/defect equilibrium reigns due to backward induction on a fixed number of games, that seems much too strong to me. Both empirically and theoretically, cooperation becomes much more plausible in iterated games.

Does the single shot game that Acemoglu and Robinson implicitly describe really seem like a better description of the situation to you? It seems very clear to me that it's not a good fit. If I had to choose between a single shot game and an iterated game, I'd choose the iterated game every time (and maybe just set the discount rate more aggressively as needed--as the post points out, we can interpret the discount rate as having to do with the probability of deposition).

Maybe the crux here is the average tenure of autocrats and who we're thinking of when we use the term?

II.

(I don't say "solve" anywhere in the post so I think the quote marks there are a bit misleading.)

I agree that to come up with something closer to a conclusion, you'd have to do something like analyze the weighted value of each of these structural factors. Even in the absence of such an analysis, I think getting a fuller list of the structural advantages and disadvantages gets us closer to the truth than a one-sided list.

Also, if we accept the claim that Acemoglu and Robinson's empirical evidence is weak, then the fact that I haven't presented any evidence on the real-world importance of these theoretical mechanisms becomes a bit less troubling. It means there's something closer to symmetry in the absence of good evidence bearing on the relative importance of structural advantages and disadvantages in each type of society.

My intuition is that majoritarian tyrannies and collective action problems are huge, pervasive problems in the contemporary world, but I won't argue for that here. I can pretty quickly come up with several examples where it might be in an autocrat's self-interest to confront coordination problems and/or majoritarian tyrannies:

• Reducing local air pollution would improve an autocrat's health
• Reducing overuse of antibiotics in animal agriculture could reduce their risk of contracting an antibiotic-resistant infection
• Allowing/encouraging immigration (for some autocratic country appealing to immigrants) could boost the economy in a way that benefits the autocrat and leads them to overrule the preferences of locals

Obviously, each of these examples is only the briefest sketch and way more work would have to be done to make things conclusive.

I have not read it, but that is an interesting conclusion from "Why Civil Defense Failed." Some of the interventions for preventing firestorming were natural gas shut off, electrical shutoff, enhanced sprinklers, and automatically closing shades. One proposed in 1967 was intentionally exploding nuclear weapons beneath the city to create a fire break to stop the spread of a mass fire-talk about fighting fire with fire! That is interesting that the political incentive might actually be greater now to prevent fire storming. Still, preparing for alternate foods would be lower cost than preventing fire storming given nuclear winter or storing more food. So that's why I think alternate foods have a better chance of actually getting implemented. Yes, the industry that could be used for alternate foods may be targeted, but some alternate foods do not depend on industry still functioning. Furthermore, if the world were prepared for alternate foods, it could potentially provide these foods to the target countries.

I was one of the non-maximizers in the poll. To expand on what I wrote there, I hold the following interpretation of this bit of Will's article:

(i) the use of evidence and careful reasoning to work out how to maximize the good with a given unit of resources, tentatively understanding ‘the good’ in impartial welfarist terms, and

(ii) the use of the findings from (i) to try to improve the world.

When I see "the use of the findings from (i)", I don't take that to necessarily mean "acting to maximize good with a high level of certainty". Instead, I interpret it as "using what we understand about maximizing good in order to make decisions that help you do more good, even if you are not actively trying to maximize your own impact".

To use an example:

Let's say that someone donates 10% of their income to AMF, because they read GiveWell's research and believe that AMF is among the best opportunities in global health/development.

Let's say that this person also hasn't carried out explicit expected-value calculations. If you ask them whether they think AMF is the best option for doing good, they say:

"Probably not. There are hundreds of plausible ways I could donate my money, and the odds that I've chosen the right one are quite low. But I don't want to spend a lot of time trying to combine my values and the available evidence to find a better option, because I don't think that I'll increase my certainty/impact enough for that use of time to feel valuable."

I still consider this person to be "practicing effective altruism". There are probably ways they could do more good even accounting for the time/energy/happiness costs of learning more. One could think of them as slightly "lazy" in their approach. Even so, they used evidence and reasoning to evaluate GiveWell's AMF writeup and determine that AMF was an unusually good option, compared to other options they knew about. They then acted on this finding to try improving the world through a donation. This feels EA-aligned to me, even if they aren't actually trying to "maximize" their impact.

There is a spectrum of "maximization", ranging from people who follow EA charity evaluators' suggestions without doing any original research to people who conduct hundreds of hours of research per year and are constantly seeking better options. I think that people in EA cover the full spectrum, so my definition of "EA" doesn't emphasize "maximizing" -- it's about "using evidence and reason to do better, even if you stop trying to do better after a certain point, as long as you aren't committing grave epistemic sins along the way".

...or something like that. I haven't yet worked this out rigorously.

My impression (also not a physicist) is that there's no obvious connection between a wave function collapsing somewhere in the universe and your neurons churning through a decision about which door you'd rather walk through. Under Many Worlds, every quantum-possible universe exists, but that doesn't mean that your experience of decision-making is equal-and-opposite distributed across those worlds. If you like the look of the right door better than the left door, then probably most of your selves will go through that door.

(If you're interested in a fictional exploration of these issues, Ted Chiang's Anxiety is the Dizziness of Freedom is excellent.)

(in no particular order)

1. The application of social movement theory to EA group building

a. The tensions between a member-organising movement (grassroots) and a centrally organised (top down) movement (early draft)

b. historical case studies of movement building to learn from (brainstorming - environmental movement)

2. Ideas to improve the presence of EA in developing countries and non-EA Hubs (editing stage)

3. Climate Change and EA

a. A research agenda for EA and climate change (early draft)

b. How to make room for climate change research in the EA movement (editing stage)

4. Career Change Resources in the EA Community Research project (research stage)

These theoretical claims seem quite weak/incomplete.

• In practice, autocrats' time horizons are highly finite, so I don't think a theoretical mutual-cooperation equilibrium is very relevant. (At minimum, the autocrat will eventually die.)
• All your suggestions about oligarchy improving the tyranny of the majority / collective action problems only apply to actions that are in the oligarchy's interests. You haven't made any case that the important instances of these problems are in an oligarchy's interests to solve, and it doesn't seem likely to me.

I appreciate your reply. I feel like I am learning a lot.

Have you ever read, Why Civil Defense Failed? The author argues civil defense failed because people took MAD literally. They believed that any attempts to stop or prevent total annihilation was utterly futile and only made things worse. They did not understand that an effective deterrence might be more effective and cheaper than guaranteeing total destruction. I think we both recognize the possibility MAD does not prevent conflicts but just makes them more deadly.

Civil defense during the cold war was scrapped because people believed it would create an incentive to build more weapons. Civil defense planners were believed to only be creating more targets. Your large scale alternate food production in factories would become targets in their arguments as well as any personnel trained to implement your plans. The author describes how the office of Civil Defense had its budget cut in the 60s even with the recent Cuban missile crisis. Only 7% of Americans made any preparations during the missile crisis because they thought total annihilation meant exactly that. So why bother? Senators believed any civil defense was just a way to prepare Americans for a war that is by definition not winnable or even survivable. I would still be interested to see your research on the nuclear winter effect if FEMA was properly funded and if their personnel were well trained.

Alternate foods are important but preventing firestorms would also stop other mass extinctions and ecological collapse. Being able to farm greatly decrease our chances of going extinct from a nuclear war. The argument can be made that now with nuclear arms treaties in place we should implement the safeguards that were not done during the cold war. This is because unchecked arms growth is less likely and we will forever live with the possibly of nuclear weapons being used.

The Russian dead hand was designed to let their leaders wait and see if an attack was real or a false alarm. They could arm their dead hand knowing that if they were hit that they could still retaliate. This is also true for SLBMs that would retaliate if they confirmed there was an attack against their countries. The submarine second strike ability and their near invincibility to a first strike attack ensures that cooler heads prevail during false alarms. This reduces the likelihood of accidental full scale war. Additionally if the codes are not correct then they would not be launched at countries and would land somewhere in the ocean.

Can the US soybean crop convert into vegetable oil? What about using powdered milk for essential fats?