EA Forum Reader

Cheers!

Hi smallsilo, 

Thank you for your feedback, it is appreciated. 

It is fair to say that a key assumption of the article is EA should cater to everyone, and therefore it should also cater to women. 

My central argument is not that there is stark lack of women throughout EA (conversely, I recognise for example that CEA notes that from 2017-2020, their staff gender balance has been roughly equal between women and men). However, there do appear to be a stark lack of women at the front and centre of EA who e.g., write key books. It is also clear that EAs are still disproportionately male. 

The main point that I'm hoping to convey is that there are women in EA, but that they are not necessarily being catered for. That is to say, that in theory you could have an EA community comprised 100% of women, but if the content is not cognisant of their needs (i.e., advice is not tailored to them, or research does not consider them, when it ought to), then that in itself is not a good thing. If you do not agree with the assumption that women (and other groups) have specific needs/ considerations, then perhaps that is where our values & assumptions differ. 

I agree that the 80k framework isn't suitable for everyone, but in which case I argue that this should be made explicit.  I also agree that some of the issues presented are partially the result of broader, more complicated dynamics elsewhere - but I don't think that is an excuse not to consider or address them. Finally, I fully concur that the demographic data I present on other minorities seems like significant issue (although I would not argue that is it more so). It would be great to see some further writing on those issues. 

Thank you for letting me know about magnifymentoring, I hadn't come across them in my searches but I'll take a look and edit if required. 

Only a very boring one. No publications so far, though I continue to send things out.

Also, it's perhaps worth noting that it's unclear to me that people who enjoyed the story posted here will necessarily enjoy my other stories. I have different motives in writing different things, and I think that one consequence of this is that different stories I write might appeal to different audiences.

But in any case, if I do get something published I'll post a link here and people can decide for themselves whether they enjoy it!

Thanks for the post. I appreciate your effort to attach your reasoning to the forecasts - I think the reasoning is much more informative, especially given the huge amount of uncertainty.

A small suggestion for improvement would be to number the probabilities and quoted reasoning, so as to make it easier to match between them.

I'm willing to bet* the forum will support it - and I think it's a bad idea.

*Not, like, on the actual platform

Interesting! Do you know the content of the complaints that people were making about the existing decision-making practices?

In general, some resources I've grown from reading about related to IIDM:

• Inadequate Equilibria [? · GW]
  • How to deal with principal-agent problems, how systems get stuck in inadequate equilibria^[1], how to get out of them, when you should/shouldn't expect to be able to beat the market, and so on.
• Paul Graham's blog
  • Paul cofounded the startup-incubator Y Combinator, so most of his advice is tailored to companies much smaller than 1000, but not all of it.
  • In The Power of the Marginal he talks about the importance of leaving people slack [? · GW] in order to inspire the most effective kind of work that doesn't have to compromise [LW · GW] between several criteria.
  • Do Things That Don't Scale is mostly applicable to startups, but even medium-sized companies need to do things urgently or imperfectly sometimes.
  • In How to Lose Time and Money he posits that "fake work" is the main way people lose money because we already have hard-wired alarms for luxury expenditures but no alarms for things that look like work (virtuous, legible token-results) and doesn't actually achieve anything.
• Ben Kuhn's blog
  • Ben's one of the founders of Wave, which is now a ~1000-employee company, and his blog is gold. Alas, I think it mostly focuses on startup-culture norms, but a lot of it is still applicable.
  • Be impatient and "just deploy", err on the side of action because "time kills all deals" and attention is your scarcest resources.
• Other essential concepts:
  • Scott Garrabrant has an excellent taxonomy of various ways a system could be goodharting [LW · GW] and therefore failing to achieve its true objectives.
    • The larger the company grows, the greater the risk that it ends up goodharting on what's legible, what signals effort/due-diligence.
      • "The exertion of effort is deemed morally admirable (Studies 1–6) and is monetarily rewarded (Studies 2–6), even in situations where effort does not directly generate additional product, quality, or economic value."
  • Parallel vs serial problem analysis. Some problems that are inherently serial cannot be sliced up and distributed to several employees. Effective institutional leadership means recognising which problems are and aren't parallelisable, slicing them up to make them more so, and avoiding deadlock as a result of inefficiently coordinated program slices. There's just a bunch of computer science related to parallel processing that's applicable for thinking about institutional design (not applying them blindly, of course).
  • Technical debt, although if you're a software company you're probably well aware already. The point is just that technical debt (or "coordination debt") is very applicable to institutions as well, because the activation cost to new coordination schemes rapidly increases the larger the company grows.

1. ^{^}

   This is just a Nash equilibrium where everyone has incentives to perpetuate the status quo. Specifically, it's an equilibrium that the system is stuck with even if everyone learned that there existed a highly-certain-but-hypothetical Nash equilibrium that's better for everyone--they just can't easily switch to it because it requires simultaneous coordination.^[2]

2. ^{^}

   Thus we have the concept of "coordination activation energy/thresholds" which is an one-time upfront cost you have to pay to reach a higher Nash equilibrium. Perhaps the best and severely under-utilised tool I know of in order to overcome activation thresholds for coordination is the idea of assurance contracts [? · GW] (wikipedia).

   An Assurance Contract is a contract of the form "I commit to X if Y other people do the same". for example, "I commit to come to a protest if 100K other people make the same commitment". if less than 100K sign this contract, it has no effect. if 100K or more sign it, it goes into effect and everyone who signed it is expected to come to the protest.

   If any of the institutional problems the company faces is the result of an inadequate equilibria that can be plausibly be addressed with an assurance contract, I'd be happy to help put you in touch with people who know more about it and (optionally) share my ideas for how to practically go about it.

If Climate Villains were unlikely to counterfactually receive any representation, then the case for representing them might be stronger. But in reality, these are some of the best capitalized and politically powerful actors on the planet. They don't need our help, and will be able to find good lawyers regardless.

I actually think there is a good market based moral case for each individual to not represent them, especially if this becomes a widespread issue. This is because even if there is a pro tanto duty to ensure they are represented, there is no duty to ensure they are represented cheaply. If few lawyers are willing to defend them, the price for this will increase and they will be less inclined to pursue actions that will increase the likelihood of lawsuits. Second, the quantity and quality of lawyers they and other bad actors can hire will eventually decline, or at least the price will rise significantly, making it less likely that they can successfully pursue typical corporate tactics like overwhelming the other party with near-spurious claims and appealing indefinitely. 

On a related note, has someone looked into the cost-effectiveness of funding new podcasts vs. convincing mainstream ones to produce more impactful content, similarly to how OpenPhil funded Kurzgesagt?

For instance, has anyone tried to convince people like Lex Fridman who has already interviewed MacAskill and Bostrom, to interview more EA-aligned speakers?

My current analysis gives roughly an audience of 1-10M per episode for Lex, and I'd expect that something around $20-100k per episode would be enough of an incentive.

In comparison, when giving $10k to start a podcast, the potential reach is maybe 100-10k per episode after 10 episodes, but maybe the EV is higher because most of the impact is after those 10 first episodes. Also, the core audience is more willing to update their models and benefit from the podcast than eg the average Lex Fridman listener.

Another counterargument woud be that Lex already interviews people like MacAskill and Bostrom so the marginal impact of an additional one is low, and EA-aligned impactful speakers already manage to get on mainstream media to do outreach anyway (eg Will going on cable cable TV for WWOTF).

Great talk. I think it breaks down the problem of AI alignment well. It also reminds me of the more recent breakdown by Dan Hendryks which decomposes ML safety into three problems: robustness, monitoring and alignment.

I've noticed that a lot of good ideas seem to come from talks. For example, Richard Hamming's famous talk on working on important problems. Maybe there should be more of them.

Thanks for writing this post, it's a very succinct way to put it (I've struggled to formulate and raise this question with the AI community). 

My personal opinion is that AI research - like many other fields - relies on "the simplest definition" of concepts that it can get away with for those notions that lie outside the field.  This is not a problem in itself, as we can't all be phds in every field (not that this would solve the problem). However, my view is there are many instances where AI research and findings rely on axioms - or yield results - that require specific interpretations of concepts (re: intelligence, agency, human psychology, neuroscience etc.) that have a speculative or at least "far-from-the-average" interpretation. This is not helped by the fact that many of these terms do not always have consensus in their own respective fields.  I think that when presenting AI ideas and pitches many overlook the nuance required to formulate/explain AI research given such assumptions. This is especially important for AI work that is no longer theoretical or purely about solving algorithmic or learning problems - but extends to other scientific fields and the broader society (e.g. AI-safety).