Posts

How to estimate the EV of general intellectual progress 2020-01-27T10:21:11.076Z · score: 34 (11 votes)
What are words, phrases, or topics that you think most EAs don't know about but should? 2020-01-21T20:15:07.312Z · score: 21 (10 votes)
Best units for comparing personal interventions? 2020-01-13T08:53:12.863Z · score: 16 (5 votes)
Predictably Predictable Futures Talk: Using Expected Loss & Prediction Innovation for Long Term Benefits 2020-01-08T22:19:32.155Z · score: 11 (4 votes)
[Part 1] Amplifying generalist research via forecasting – models of impact and challenges 2019-12-19T18:16:04.299Z · score: 53 (16 votes)
[Part 2] Amplifying generalist research via forecasting – results from a preliminary exploration 2019-12-19T16:36:10.564Z · score: 31 (11 votes)
Introducing Foretold.io: A New Open-Source Prediction Registry 2019-10-16T14:47:20.752Z · score: 48 (24 votes)
What types of organizations would be ideal to be distributing funding for EA? (Fellowships, Organizations, etc) 2019-08-04T20:38:10.413Z · score: 31 (10 votes)
Conversation on forecasting with Vaniver and Ozzie Gooen 2019-07-30T11:16:23.576Z · score: 36 (11 votes)
What new EA project or org would you like to see created in the next 3 years? 2019-06-11T20:56:42.687Z · score: 72 (36 votes)
Impact Prizes as an alternative to Certificates of Impact 2019-02-20T21:25:46.305Z · score: 34 (11 votes)
Discussion: What are good legal entity structures for new EA groups? 2018-12-18T00:33:16.620Z · score: 14 (6 votes)
Current AI Safety Roles for Software Engineers 2018-11-09T21:00:23.318Z · score: 11 (5 votes)
Prediction-Augmented Evaluation Systems 2018-11-09T11:43:06.088Z · score: 6 (2 votes)
Emotion Inclusive Altruism vs. Emotion Exclusive Altruism 2016-12-21T01:40:45.222Z · score: 2 (4 votes)
Ideas for Future Effective Altruism Conferences: Open Thread 2016-08-13T02:59:02.685Z · score: 2 (4 votes)
Guesstimate: An app for making decisions with confidence (intervals) 2015-12-30T17:30:55.414Z · score: 41 (43 votes)
Is there a hedonistic utilitarian case for Cryonics? (Discuss) 2015-08-27T17:50:36.180Z · score: 9 (11 votes)
EA Assembly & Call for Speakers 2015-08-18T20:55:13.854Z · score: 8 (8 votes)
Deep Dive with Matthew Gentzel on Recently Effective Altruism Policy Analytics 2015-07-20T06:17:48.890Z · score: 4 (3 votes)
The first .impact Workathon 2015-07-09T07:38:12.143Z · score: 6 (6 votes)
FAI Research Constraints and AGI Side Effects 2015-06-07T20:50:21.908Z · score: 2 (2 votes)
Gratipay for Funding EAs 2014-12-24T21:39:53.332Z · score: 5 (5 votes)
Why "Changing the World" is a Horrible Phrase 2014-12-24T00:41:50.234Z · score: 7 (7 votes)

Comments

Comment by oagr on Any response from OpenAI (or EA in general) about the Technology Review feature on OpenAI? · 2020-02-24T15:53:35.895Z · score: 24 (13 votes) · EA · GW

I think these comments could look like an attack on the author here. This may not be the intention, but I imagine many may think this when reading it.

Online discussions are really tricky. For every 1000 reasonable people, there could be 1 who's not reasonable, and who's definition of "holding them accountable" is much more intense than the rest of ours.

In the case of journalists this is particularly selfishly-bad; it would be quite bad for any of our communities to get them upset.

I also think that this is very standard stuff for journalists, so I really don't feel the specific author here is particularly relevant to this difficulty.

I'm all for discussion of the positives and weaknesses of content, and for broad understanding of how toxic the current media landscape can be. I just would like to encourage we stay very much on the civil side when discussing individuals in particular.

Comment by oagr on Any response from OpenAI (or EA in general) about the Technology Review feature on OpenAI? · 2020-02-22T11:41:47.986Z · score: 17 (16 votes) · EA · GW

I feel like it's quite possible that the headline and tone was changed a bit by the editor, it's quite hard to tell with articles like this.

I wouldn't single out the author of this specific article. I think similar issues happen all the time. It's a highly common risk when allowing for media exposure, and a reason to possibly often be hesitant (though there are significant benefits as well).

Comment by oagr on How to estimate the EV of general intellectual progress · 2020-02-11T17:27:51.451Z · score: 2 (1 votes) · EA · GW

Agreed, though the suggestions are appreciated!

VOI calculations in general seem like a good approach, but figuring out how to best apply them seems pretty tough.

Comment by oagr on Request for Feedback: Draft of a COI policy for the Long Term Future Fund · 2020-02-06T22:09:14.343Z · score: 14 (6 votes) · EA · GW

I'm a bit surprised that recusal seems to be pushed for last-resort in this document. Intuitively I would have expected that because there are multiple members of the committee, many in very different locations, it wouldn't be that hard to have the "point of contact" be different from the "one who makes the decision." Similar to how in some cases if one person recommends a candidate for employment, it can be easy enough to just have different people interview them.

Recusal seems really nice in many ways. Like, it would also make some things less awkward for the grantors, as their friends wouldn't need to worry about being judged as much.

Any chance you could explain a bit how the recusal process works, and why it's preferred to not do this? Do other team members often feel really unable to make decisions on these people without knowing them? Is it common that the candidates are known closely by many of the committee members, such that collective recusal would be infeasible?

Comment by oagr on Request for Feedback: Draft of a COI policy for the Long Term Future Fund · 2020-02-06T21:54:05.597Z · score: 4 (2 votes) · EA · GW

Kudos for writing up a proposal here and asking for feedback publicly!

Companies and nonprofits obviously have boards for similar situations, these funds having similar boards that would function in similar ways would seem pretty reasonable to me. I imagine it may be tricky to find people both really good and really willing. Having a board kind of defers some amount of responsibility to them, and I imagine a lot of people wouldn't be excited to gain this responsibility.

I guess one quick take would be that I think the current proposed COI policy seems quite lax, and I imagine potential respected board members may be kind of uncomfortable if they were expected to "make it respectable". So I think a board may help, but wouldn't expect it help that much, unless perhaps they did some thing much more dramatic, like work with the team to come up with much larger changes.

I would personally be more excited about methods of eventually having the necessary resources to be able to have a less lax policy without it being too costly; for instance, by taking actions to grow the resources dedicated to funding allocations. I realize this is a longer-term endeavor, though.

Comment by oagr on EA Forum Prize: Winners for December 2019 · 2020-02-01T11:41:08.893Z · score: 2 (1 votes) · EA · GW

Would it have been reasonable for you to have been secretively part of the process or something?

Some options:

  1. You write in that if you win, you just don't accept the cash prize.
  2. You write in that if you win, they tell you, but don't tell anyone else, and select the next best person for the official prize.

I'd be curious what the signaling or public value of the public explanation, "Person X would have won 1st place, but removed themselves from the running" would be compared to "Person X won 1st place, but gave up the cash prize"

Comment by oagr on Is learning about EA concepts in detail useful to the typical EA? · 2020-02-01T10:35:46.237Z · score: 3 (2 votes) · EA · GW

Quick take:

I think that in theory, if things were being done quite well and we had a lot of resources, we should be in a situation where most EAs really don't need much outside of maybe 20-200 hours of EA-specific information; after which focusing more on productivity and career-specific skills would result in greater gains.

Right now things are more messy. There's no great one textbook, and the theory is very much still in development. As such, it probably does require spending more time, but I'm not sure how much more time.

I don't know if you consider these "EA" concepts, but I do have a soft spot for many things that have somewhat come out of this community but aren't specific to EA. These are more things I really wish everyone knew, and they could take some time to learn. Some ideas here include:

  • "Good" epistemics (This is vague, but the area is complicated)
  • Bayesian reasoning
  • Emotional maturity
  • Applied Stoicism (very similar to managing one's own emotions well)
  • Cost-benefit analyses and related thinking
  • Pragmatic online etiquette

If we were in a culture that was firmly attached to beliefs around the human-sacrificing god Zordotron, I would think that education to carefully remove both the belief and many of the practices that are caused by that belief, would be quite useful, but also quite difficult. Doing so may be decently orthogonal to learning about EA, but would seem like generally a good thing.

I believe that common culture taught in schools and media is probably not quite as bizarre, but definitely substantially incorrect in ways that are incredibly difficult to rectify.

Comment by oagr on Seeking Advice: Arab EA · 2020-01-30T20:21:21.241Z · score: 4 (3 votes) · EA · GW

Sounds good, best of luck with that! Writing posts on the EA forum or LessWrong on things you find interesting and partaking in the conversation can be a good way of getting up to speed and getting comfortable with ongoing research efforts.

Comment by oagr on Seeking Advice: Arab EA · 2020-01-30T20:18:58.616Z · score: 11 (8 votes) · EA · GW

I just want to point out that this seems very, very difficult to me, and I would not recommend trusting "being safe" unless you really have no other choice.

I know of multiple very smart people who have tried to stay anonymous, got caught, and bad things happened. (For instance, read many books on "top hackers")

Comment by oagr on Defining Effective Altruism · 2020-01-30T11:38:53.114Z · score: 3 (1 votes) · EA · GW

After more thought in areas of definition, I've come to believe that the presumption of authority can be a bit misrepresentative.

I'm all for the coming up and encouraging of definitions of Effective Altruism and other important topics, but the phrase "The definition of effective altruism" can be seen to presuppose authority and unanimity.

I'm sure that even after this definition was proposed, alternative definitions will be used.

Of course if there were to be one authority on the topic it would be William MacAskill, but I think that even if there were only one main authority, the use of pragmatic alternative definitions could only be discouraged. It would be difficult to call them incorrect or invalid. Dictionaries typically follow use, not create it.

Also, to be clear, I have this general issue with a very great deal of literature, so it's not like I'm blaming this piece because it's particularly bad, but rather, I'm pointing it out because this piece is particularly important.

Maybe there could be a name like the "The academic definition...", "The technical definition", or "The definition according to the official CEA Ontology". Sadly these still use "The" which I'm hesitant to, but they are at least more narrow.

Comment by oagr on Seeking Advice: Arab EA · 2020-01-29T20:59:03.552Z · score: 16 (11 votes) · EA · GW

Also, I'm not sure if you used a pseudonym here, but I hope you did. I'd suggest being pretty wary of posting some details like that online, I would imagine they could easily be found later on if you did anything along those lines.

Comment by oagr on Seeking Advice: Arab EA · 2020-01-29T20:57:17.600Z · score: 32 (18 votes) · EA · GW

I would quite strongly recommend against lying and also strongly would recommend against anything that could give you the death penalty! I realize EAs are kind of intense, but that seems far over the line. Please do not engage inactivities that could directly put yourself in danger.

It sounds like you're in a challenging situation. I could definitely empathize with that and for others in similar situations.

One thing to consider may be that if creating an EA group may come with so many local disadvantages, it may mean that even once you do create it, it may continue to have severe limits to growth, limiting its potential anyway.

I think I'd encourage you to feel fine not contributing anything useful right away, but instead focus efforts on your long-term efforts. Learning a lot and doing work to secure a great career can go a long way, though it will of course take a while and can be frustrating for that reason. I know 80,000 Hours has written a lot about topics related to this.

Comment by oagr on How to estimate the EV of general intellectual progress · 2020-01-28T11:21:52.353Z · score: 4 (3 votes) · EA · GW

Thanks! This is interesting, will spend some time thinking about.

  1. Please don't worry much about embarrassing yourself! It's definitely a challenge with forums like this, but it would be pretty unreasonable for anyone to expect that post/comment authors have degrees in all the possibly relevant topics.
  2. Low-effort thoughts can be pretty great, they may be some of the highest value-per-difficulty work.
Comment by oagr on How to estimate the EV of general intellectual progress · 2020-01-28T11:18:01.573Z · score: 2 (1 votes) · EA · GW

Definitely agreed.

Comment by oagr on How to estimate the EV of general intellectual progress · 2020-01-28T11:04:26.052Z · score: 3 (2 votes) · EA · GW

Nice find! This seems like a useful step, though of course likely considerably different than what I imagine consequentialists would aim for.

Comment by oagr on Doing good is as good as it ever was · 2020-01-23T10:36:39.862Z · score: 18 (10 votes) · EA · GW

I think that personally, I'd mostly advocate for attempts to decouple motivation from total impact magnitude, rather for attempts to argue that high impact magnitude is achievable, so to speak, when trying to improve motivation.

If you attach your motivations to a specific magnitude of "$2,000 per life saved", then you can expect them to fluctuate heavily when estimates change. But ideally, you would want your motivations to stay pretty optimal and thus consistent for your goals. I think this ideal is somewhat possible and can be worked towards.

The main goal of a consequentialist should be to optimize a utility function, it really shouldn't matter what the specific magnitudes are. If the greatest thing I could do with my life is to keep a small room clean, then I should spend my greatest effort on that thing (my own wellbeing aside).

I think that most people aren't initially comfortable with re-calibrating their goals for arbitrary utility function magnitudes, but I think that learning to do so is a gradual process that could be learned, similar to learning stoic philosophy.

It's similar to learning how to be content no matter one's conditions (aside from extreme physical ones), as discussed in The Myth of Sisyphus.

https://en.wikipedia.org/wiki/The_Myth_of_Sisyphus

Comment by oagr on Potential downsides of using explicit probabilities · 2020-01-23T09:35:16.953Z · score: 2 (1 votes) · EA · GW

I think that makes sense. Some of it is a matter of interpretation.

From one perspective, the optimizer's curse is a dramatic and challenging dilemma facing modern analysis. From another perspective, it's a rather obvious and simple artifact from poorly-done estimates.

I.E. they sometimes say that if mathamaticians realize something is possible, they consider the problem trivial. Here the optimizer's curse is considered a reasonably-well-understood phenomena, unlike some other estimation-theory questions currently being faced.

Comment by oagr on Potential downsides of using explicit probabilities · 2020-01-21T22:55:30.388Z · score: 3 (2 votes) · EA · GW

And I think that, even when one is extremely uncertain, the optimizer’s curse doesn’t mean you should change your preference ordering (just that you should be far less certain about it, as you’re probably greatlyoverestimating the value of best-seeming option).

Ok, I'll flag this too. I'm sure there are statistical situations where an extreme outcome implies that an adjustment for correlation goodharting would make it seem worse than other options; i.e. change order.

That said, I'd guess this isn't likely to happen that often for realistic cases, especially when there aren't highly extreme outliers (which, to be fair, we do have with EA).

I think one mistake someone could make here would be to say that because the ordering may be preserved, the problem wouldn't be "fixed" at all. But, the uncertainties and relationships themselves are often useful information outside of ordering. So a natural conclusion in the case of intense noise (which leads to the optimizer's curse) would be to accept a large amount of uncertainty, and maybe use that knowledge to be more conservative; for instance, trying to get more data before going all-in on anything in particular.

Comment by oagr on What are words, phrases, or topics that you think most EAs don't know about but should? · 2020-01-21T20:37:21.267Z · score: 7 (4 votes) · EA · GW

The Cooperative Principle

The cooperative principle describes how people achieve effective conversational communication in common social situations—that is, how listeners and speakers act cooperatively and mutually accept one another to be understood in a particular way.

There are 4 corresponding maxims. I think the main non-obvious ones are:

Maxim of quantity:

  1. Make your contribution as informative as is required (for the current purposes of the exchange).
  2. Do not make your contribution more informative than is required.

Maxim of relevance

  1. Be relevant to the discussion. (For instance, when responding to, "What would you like for lunch" and you respond "I would like a sandwhich"; you are expected to be responding to that very question, not to be making an unrelated statement.)

I think this video explains this well.

Why this is interesting
I've definitely been in conversations where bringing up maxims of quantity and relevance would have been useful to bring up. Conversation and discussion can be quite difficult. We do a lot of that.

Comment by oagr on What are words, phrases, or topics that you think most EAs don't know about but should? · 2020-01-21T20:30:12.181Z · score: 16 (6 votes) · EA · GW

Consilience

The principle that evidence from independent, unrelated sources can "converge" on strong conclusions

This word can arguably be used to describe the "Many Weak Arguments" aspect of the "Many Weak Arguments vs. One Relatively Strong Argument" post. JonahSinick pointed that out in that post.

Why this is interesting
Consilience is important for evaluating claims. There's a fair bit of historic discussion and evidence now that shows how useful it can be to get a variety of evidence from many different sources.

Comment by oagr on What are words, phrases, or topics that you think most EAs don't know about but should? · 2020-01-21T20:22:06.019Z · score: 6 (4 votes) · EA · GW

Deepity

The term refers to a statement that is apparently profound but actually asserts a triviality on one level and something meaningless on another. Generally, a deepity has (at least) two meanings: one that is true but trivial, and another that sounds profound, but is essentially false or meaningless and would be "earth-shattering" if true.

Why this is interesting
I mostly think this is just a great phrase to describe a lot of difficult language I occasionally see get used in moral discussions.

Comment by oagr on What are words, phrases, or topics that you think most EAs don't know about but should? · 2020-01-21T20:18:25.041Z · score: 7 (5 votes) · EA · GW

Nobel Cause Corruption

From Wikipedia:

Noble cause corruption is corruption caused by the adherence to a teleological ethical system, suggesting that people will use unethical or illegal means to attain desirable goals, a result which appears to benefit the greater good. Where traditional corruption is defined by personal gain, noble cause corruption forms when someone is convinced of their righteousness, and will do anything within their powers to achieve the desired result. An example of noble cause corruption is police misconduct "committed in the name of good ends" or neglect of due process through "a moral commitment to make the world a safer place to live."

Why this is interesting
I think one serious concern around consequentialist thought is that it can be used in dangerous ways. I think this term describes some of this, and the corresponding literature provides examples that seem similar to what I can expect future people to follow who misuse EA content.

Comment by oagr on Potential downsides of using explicit probabilities · 2020-01-21T11:49:02.815Z · score: 3 (2 votes) · EA · GW

Hm... Some of this would take a lot more writing than would make sense in a blog post.

On overconfidence in probabilities vs. intuitions: I think I mostly agree with you. One cool thing about probabilities is that they can be much more straightforwardly verified/falsified and measured using metrics for calibration. If we had much larger systems, I believe we could do a great deal of work to better ensure calibration with defined probabilities.

"should eventually be used for most things"

I'm not saying that humans should come up with unique probabilities for most things on most days. One example I'd consider "used for most things" is a case where an AI uses probabilities to tell humans which actions seem the best, and humans go with what the AI states. Similar could be said for "a trusted committee" that uses probabilities as an in-between.

"we could learn to get much better than them later"

I think there are strong claims that topics like Bayes, Causality, Rationality even, are still relatively poorly understood, and may be advanced a lot in the next 30-100 years. As we get better with them, I predict we would get better at formal modeling.

I reject expected utility theory (and related stances such as cost-benefit analysis), at least if it comes as a formal way of spelling out a maximizing consequentialist moral stance which does not properly incorporate rights.

This is a complicated topic. It think a lot of Utilitarians/Consequentialists wouldn't deem many interpretations of rights as metaphysical or terminally-valuable things. Another way to look at it would be to attempt to map the rights to a utility function. Utility functions require very, very few conditions. I'm personally a bit cynical of values that can't be mapped to utility functions, if even in a highly-uncertain way.

But Chris Smith quotes the proposed solution, and then writes... It's clear Chris Smith has thought about some of this topic a fair bit, but my impression is that I disagree with him. It's quite possible that much of the disagreement is semantic; where he says 'this solution is unworkable' I may say, 'the solution results in a very wide amount of uncertainty'. I think it's clear to everyone (the main researchers anyway) that there's little data about many of these topics, and that Bayesian or any kind of statistical manipulations can't fundamentally convert "very little data" into "a great deal of confidence".

Kudos for identifing that post. The main solution I was referring to was the one described in the second comment:

In statistics the solution you describe is called Hierarchical or Multilevel Modeling. You assume that you data is drawn from a set of distributions which have their parameters drawn from another distribution. This automatically shrinks your estimates of the distributions towards the mean. I think it's a pretty useful trick to know and I think it would be good to do a writeup but I think you might need to have a decent grasp of bayesian statistics first.

The optimizer's curse arguably is basically within the class of Goodhart-like problems https://www.lesswrong.com/posts/5bd75cc58225bf06703754b2/the-three-levels-of-goodhart-s-curse

I'm not saying that these are easy to solve, but rather, there is a mathematical strategy to generally fix them in ways that would make sense intuitively. There's no better approach than to try to approximate the mathematical approach, or go with an approach that in-expectation does a decent job at approximating the mathematical approach.

Comment by oagr on Love seems like a high priority · 2020-01-21T10:58:18.113Z · score: 3 (2 votes) · EA · GW

Just want to second this, I think it's a pretty well-known issue in the industry. Dating apps that do incredibly well at setting up people on dates will get little use (because they typically charge monthly rates, and will get to charge fewer months if the customer is happy and leaves quickly). It's possibly a very large market failure.

I could imagine a hypothetical app that was able to charge a lot more initially (Like, $2,000) but did a much better job. Of course, one issue with this is that these apps need a lot of users, so this could be really difficult.

I think it could be possible to figure out a solution here, but I imagine the solution may be 2/3rds payment/economic-innovation.

Perhaps an idea solution would look something like a mix between personal guidance and online support.

Comment by oagr on Potential downsides of using explicit probabilities · 2020-01-20T14:22:07.835Z · score: 9 (5 votes) · EA · GW

Kudos for this write-up, and for your many other posts (both here and on LessWrong, it seems) on uncertainty.

Overall, I'm very much in the "Probabilities are pretty great and should eventually be used for most things" camp. That said, I think the "Scout vs. Soldier" mindset is useful, so to speak; investigating both sides is pretty useful. I'd definitely assign some probability to being wrong here.

My impression is that we're probably in broad agreement here.

Some quick points that come to mind:

  1. The debate on "are explicit probabilities useful" is very similar to those of "are metrics useful", "are cost-benefit analyses useful", and "is consequentialist reasoning useful." I expect that there's broad correlation between those who agree/disagree with these.

  2. In cases where probabilities are expected to be harmfully, hopefully probabilities could be used to tell us as such. Like, we could predict that explicit and public use would be harmful.

  3. I'd definitely agree that it's very possible to use probabilities poorly. I think a lot of Holden's criticisms here would fall into this camp. Neural Nets for a while were honestly quite poor, but thankfully that didn't lead to scientists abandoning those. I think probabilities are a lot better now, but we could learn to get much better than them later. I'm not sure how we can get much better without them.

  4. The optimizer's curse can be adjusted for with reasonable use of Bayes. Bayesian hierarchical models should deal with it quite well. There's been some discussion of this around "Goodhart" on LessWrong.

Comment by oagr on Love seems like a high priority · 2020-01-19T22:26:29.802Z · score: 6 (4 votes) · EA · GW

I think one possible point against tractability is that many countries have been trying to work on similar things as a means to increase population.

However, the flip side to this is that if you were to come up with a clever intervention, these countries could act as large non-EA buyers.

I remember seeing some other writing on dating apps in general. I think among smart tech people I know there is some agreement that dating apps could be improved a whole lot. That said, it is difficult to make money from them, so it's not incredibly exciting as a business venture.

Some possibly relevant links:

https://www.quora.com/Are-there-countries-which-encourage-their-population-to-increase

https://www.economist.com/asia/2019/10/03/japans-state-owned-version-of-tinder

https://www.businessinsider.com/japanese-government-dating-services-2016-10?r=US&IR=T

https://www.vox.com/2016/5/23/11440638/child-benefit-child-allowance

https://www.mentalfloss.com/article/33485/6-creative-ways-countries-have-tried-their-birth-rates

https://www.dw.com/en/france-moves-to-encourage-large-families/a-1720921

Comment by oagr on Love seems like a high priority · 2020-01-19T20:31:00.903Z · score: 19 (10 votes) · EA · GW

Kudos for seriously recommending a novel-to-EA-for-me intervention! It seems there have been a few relatively novel ideas posted recently, it's been quite nice. I think the base rate for these sorts of interventions making it through all the next filters (much more EA consideration, actual implementation) is quite low, and I don't have particular reason to currently believe this is much different than other recommendations. That said, even if there were a 0.1% chance this were cost-effective, the EV of introducing it could be quite high.

Comment by oagr on The 'sprinting between oases' strategy · 2020-01-17T21:24:58.931Z · score: 4 (2 votes) · EA · GW

I think I like the idea of more abstract posts being on the EA Forum, especially if the main intended eventual use is straightforward EA causes. Arguably, a whole lot of the interesting work to be done is kind of abstract.

This specific post seems to be somewhat related to global stability, from what I can tell?

I'm not sure what the ideal split is between this and LessWrong. I imagine that as time goes one we could do a better cluster analysis.

Comment by oagr on Space governance is important, tractable and neglected · 2020-01-16T07:08:26.521Z · score: 25 (10 votes) · EA · GW

+1 for a serious suggestion of an intervention that to me is pretty novel to EA. I feel like we should be considering a larger class of options than we typically have. Space governance seems like a reasonable option to consider to me.

Comment by oagr on Moloch and the Pareto optimal frontier · 2020-01-14T20:16:40.271Z · score: 3 (2 votes) · EA · GW

I like the first bit, but am a bit confused on the Moloch bit. Why exactly would we expect that it "maximizes competitiveness"?

Comment by oagr on Disadvantages of the measures · 2020-01-13T08:37:15.814Z · score: 2 (1 votes) · EA · GW

I guess one question is:

Can we use meta-metrics to tell us the expected value of using specific metrics?

There are definitely times where specific metrics are EV-negative, but I'd hope that can be determined with other metrics. (Like a forecast of the estimated value)

Comment by oagr on Should and do EA orgs consider the comparative advantages of applicants in hiring decisions? · 2020-01-13T08:34:17.358Z · score: 16 (7 votes) · EA · GW

On Rejection

I'd be quite surprised if rejection were frequent. I've been part of / close to a few organizations and haven't heard it discussed. A few reasons:

  1. I think most orgs are overconfident in themselves compared to other organizations (probably due to selection effects).
  2. I think most organizations would prefer to give the applicants the option, rather than removing it. They may tell the applicant in question that they believe it's possible that their job is less effective, but that it's still the applicant's decision to choose.
  3. "It feels wrong" in similar ways that lying or stealing feel wrong.
  4. I haven't heard about this being a factor, but I would imagine there could be legal issues for rejecting a candidate because you want to effectively control which group they work for.

I think (2) is a pretty good reason, for humility reasons if nothing else. Applicants may have a lot of good reasons for doing things that may at first seem sub-optimal. (3) is also pretty good. I could imagine instances where there's one highly-anxiety-producing but seemingly-effective option for a person. It seems kind of cruel if all their other job prospects refuse them to force them into that position. I could imagine some pretty nasty decision consequences that could result if that were a consideration.

On Recommendations to Other Orgs

I have witnessed cases of organizations suggesting good people to other organizations, especially if they both (1) think they were good, but (2) didn't fit with their own application process. This seems pretty reasonable to me.

Personally, when I chat to potential hires of things I'm working around, I try to be as honest as possible regarding to which organizations they would be best for, even sometimes connecting them with people representing other organizations.

In some sense these organisations are competing, but in a bigger sense, we're all trying to help make sure the world goes OK.

Comment by oagr on Long-Term Future Fund: November 2019 short grant writeups · 2020-01-12T12:35:23.582Z · score: 16 (8 votes) · EA · GW

Thanks so much for the thoughtful response. My guess is that you have more information than I do, and are correct here, but just in case I wanted to share some thoughts that I have had that would provide some counter-evidence.

Don't feel the need to respond to any/all of this, especially if it's not useful to you. The LTFF would be the group who would hypothetically be taking this advice (If I were fully convinced of your side, it wouldn't make a big difference, because I'm not doing much grantmaking myself).

First, that clarification is useful about Roam, and my main comment wasn't really about them in particular, but rather them as an example. I'd definitely agree that if they are getting VC funding, then LTFF funding is much less important. I'm quite happy to hear that they raised their seed round![1]

I do think that, right now, at the margin, small interventions are particularly underfunded.

From my position, now, I don't quite get this sense, just FYI. My impression is that the "big donor" funding space is somewhat sporadic and not very forthcoming in their decisions and analysis with possible donees. There are several mid-sized EA-related organizations I know that are getting very little or very sporadic (i.e. 1-off grant) funding from OpenPhil or similar without a very clear picture on how that will translate to the long term. OpenPhil itself generally only donates 50% revenue maximum, and it's not always clear who is best to make the rest of that. It's also of course not very preferable to rely on one or two very large donors.

Having a few dependable medium-sized donors (like Larks) is really useful to a bunch of mid-sized EA orgs. Arguably the EA Funds could effectively be like these medium-sized donors, and I could see that being similarly useful.

I hate to bring up anonymous anecdotal data, but around half of the EA orgs I'm close with (of around 7) seem to have frustrations around the existing funding situation. These are 3-40 person organizations.

Some other thoughts:

  1. The Meta Fund seems to primarily give to medium-sized groups. I'm not convinced they are making a significant mistake, and I don't feel like they are so different from the Long-Term Future Fund that this decision should obviously be different.
  2. Many of the Meta Fund's payouts (and likely those of other funds, theirs just come to mind) are to groups that then re-grant that money. The EA Community Building Grants work that way, and arguably Charity Entrepreneurship is similar. I think that to me, having an EA group give money directly to people, rather than having it first go through a more locally-respected group that can focus more on specific types of experiments, is a bit of an anti-pattern; it's important when no group yet exists, but is generally sub-ideal. I personally would guess that $40k going to either of those two groups, would be better than the Meta Fund having given that money directly to individuals in similar reference classes (community builders and potential entrepreneurs). Having the in-between group allows for more expertise of the granter and more specialized mentorship, resources, and reputability.
  3. Similar to (2), I'm in general more comfortable with the idea of "give money to an org to hire a person" than the idea of "give money to one person to do independent work." The former comes with a bunch of benefits, similar to what was mentioned in (2).

I guess, backing up a bit, that there's quite a few reasons for organizations (perhaps small ones) to generally be more efficient than individuals on their own. That said, it's quite possible that while this may be the case, the organizations are far less neglected, and as of right now many of the individual projects are more neglected as to make them worthwhile.

I think I could buy this up to some amount of money; the amount isn't clear. Maybe if the LTFF gets to $4mil/year it should start moving to larger things?

Related, I just checked, and it seems like the 3 other EA Fund groups all primarily have been giving to existing medium-sized orgs. It's not obvious to me that Long-Term Future topics are dramatically different. There are big donors for those 3 categories as well, though maybe with less funding. Of course, an inconsistency could mean that they should consider changing that idea instead.

Larks also brought up another argument for the LTFF focusing on smaller projects in his last AI Alignment Review:

Larks seems like the ideal model of a medium-sized donor who's willing to spend time evaluating the best nonprofits. I'd imagine most other donors of their contribution amount are less familiar than they are. Personally, if I were in Lark's position, I'd prefer my money go through an interim body like the Long-Term Future Fund even when it goes to medium-sized organizations, because I wouldn't trust my own biases, but Larks may be particularly good. Another possible benefit of the LLFF is that fewer donors could be just easier to work with.

I imagine that there are many other donors who either/both would trust a dedicated group more, or just don't have enough money for it to be really worth the time to select (and decide against the lotteries for various reasons). Another obvious possible solution here would be to have a separate long-term fund for donors who do want to use it for medium-sized groups.

I could of course be wrong here! This is an empirical question about what the current and possible donors would want to see.

[1] I'd separately note though that I'm a bit uncomfortable about "tech things being used by EAs" to try going the VC-round, in general. The levels of scale needed for such things may require sacrifices that would make tools much less good for EA purposes; I definitely got this sense when working on Guesstimate. It may be that Roam is one of the super fortunate ones to both be able to do good for EA groups and also have a reasonable time raising money, but I think this in general is really tough and don't generally recommend others try to copy this approach.

Comment by oagr on Long-Term Future Fund: November 2019 short grant writeups · 2020-01-11T10:00:01.191Z · score: 16 (7 votes) · EA · GW

Kudos for another lengthy write up!

I know some of the people here so don't want to comment on individuals. I would say though that I'm particularly excited about collaborations with the Topos Institute; they seem like one of the most exciting academic groups/movements right now, and experimenting with working with them seems pretty valuable to me. Even if the only result is that they forward smart people the way of EA problems, it could be quite beneficial.

One uncertainty; I noticed that very few of these groups were large & repeated; the main one being Roam Research, which you say you plan to stop funding. Who do you think should fund small-to-medium sized groups around long-term work? My impression was that the Long-Term Future Fund was the place for people who wanted to put money into the most cost-effective long-term future projects, but it seems like it may be somewhat limited to very small projects; which could be a very reasonable decision, but a bit non-obvious to outsiders.

Some obvious subquestions here:

  1. Are these small interventions mostly more cost-effective than larger ones?
  2. If (1) is not true, then what are the best current strategies for funding a mix of small and larger interventions? Is the expectation that large donors set up individual relationships with these larger groups, and just the Long-Term Future Fund for the smaller groups?
  3. If (1) is not true, do you think it's possible that in the future EA Funds could be eventually be adjusted to also handle more of larger interventions?
Comment by oagr on What should EAs interested in climate change do? · 2020-01-10T21:39:46.812Z · score: 7 (5 votes) · EA · GW

I think personally, I'd expect that some marginal experiments could be pretty high-value for the information value (i.e. testing the waters). I'd be curious about OpenPhil's investigations into the issue and what new information, if any, they would find most useful.

Comment by oagr on What should EAs interested in climate change do? · 2020-01-10T19:03:32.162Z · score: 9 (6 votes) · EA · GW

I think the most obvious thing is prioritization; i.e. "figuring out what to do". My impression is that there's a considerable amount of interesting work to do to apply an EA-mindset to climate change and get a better sense of the opportunities and their effectiveness.

I wrote a bit that's related: https://www.lesswrong.com/posts/Df2uFGKtLWR7jDr5w/ozziegooen-s-shortform?commentId=ZT5ArGKSWsctoWrZF

Comment by oagr on In praise of unhistoric heroism · 2020-01-08T21:35:42.857Z · score: 5 (3 votes) · EA · GW

Great post!

A few points:

  1. I like the metaphor "the Game" a lot to describe consequentialism. I've been thinking about it a fair bit recently. Its use in The Wire seemed particularly relevant, and a bit less so in Game of Thrones. I obviously don't like the book association, but I think its quite fair to consider that a minor thing in comparison to its' greater use.

  2. I think the idea of "compare yourself to the best person" is a bit of a fallacy, though one that basically everyone seems to do. What really matters is that people do what is optimal; and that means comparing yourself to whatever you find pragmatically useful. The best person obviously shouldn't compare themselves only to themselves, as they should be aiming for higher still.

  3. While this may be a suboptimal funeral, I think my idea of an ideal funeral would look something like a few analysts going over my life to try to get a sense of how well I did on The Game relative to what I should have aimed for, given my very unique advantages and disadvantages. Then hopefully writing up a document of "lessons learned" for future people. Something like, "Ozzie was pretty decent at Y, and had challenges Z and Q. He tried doing these things, which produced these modest results. We'd give him maybe a C- and suggest these lessons that others could apply to make sure they do better."^1

[1] Ok, maybe this is trolling a little.

Comment by oagr on More info on EA Global admissions · 2020-01-04T15:55:21.761Z · score: 5 (3 votes) · EA · GW

This seems like it involved some difficult decisions. Thanks for selecting what sounds like it was most reasonable and explaining here.

I guess my main take-away is not anything about the details of this event, but rather the point that it seems like it's become more urgent/important for more work to be done in this area.

It seems to me like the previous few EAGs were done with relatively few people, and now there's an opportunity for a bunch more work.

I'm a bit curious; how big of a challenge does this feel for CEA, and are there ways you think community members could help out? (For instance, applying to CEA to help run similar events, or holding more local meetups, etc.)

Comment by oagr on We're Rethink Priorities. AMA. · 2019-12-16T19:50:31.774Z · score: 3 (2 votes) · EA · GW

I generally don't like negativity, including negativity about negativity! (Harsh downvotes on cynical-seeming comments).

There are other times where harsh comments get a lot of upvotes; like around Leverage Research.

I think many people think that those go a bit too far and seem a bit more intense than the individuals mean. Similar to how you didn't mean your comment to be too harsh, the downvoters probably didn't mean to be too harsh in that signal.

Comment by oagr on We're Rethink Priorities. AMA. · 2019-12-14T22:22:03.723Z · score: 11 (6 votes) · EA · GW

Good point! This does sound limiting. I guess I'd flag it, for one, as a message to funders when determining how to structure things. I'd hope that the EA Fund managers and others can be convinced to eventually donate in more and more optimal ways, if those ways really are optimal (and that can be made very clear). For instance, when donating to specific projects, paying attention to make clauses to ensure that the researchers have flexibility to make significant modifications if necessary.

Comment by oagr on We're Rethink Priorities. AMA. · 2019-12-14T22:17:59.275Z · score: 13 (6 votes) · EA · GW

Yep, makes sense. One unfortunate and frustrating thing that I've noticed over the last few years is that lots that gets posted on the forum and similar gets amplified and misunderstood by many people online.

I'm quite sure that you were pretty reasonable, but I would flag that I would guess that at least some readers wouldn't understand the nuance, and may just think something like, "I guess this user is using this as a sarcastic-like take at saying they think that Rethink's work is low quality." When I read your post I personally had a lot of uncertainty on where exactly you were coming from.

I'd definitely encourage you to keep on pointing things out and would probably recommend not modifying the main messages, buy may suggest that you be extra careful with the wording on such items. It kind of sucks, but that seems like one restriction of public forums like this.

Similar, to be extra clear, so this doesn't get misinterpreted, my points are:

  1. Feedback and comments are great! Please leave more! Don't censor your main points, especially if they are important!
  2. However, if they are things that could be misunderstood by some audience members in fairly impactful ways, try on the margin to make things extra clear.
Comment by oagr on We're Rethink Priorities. AMA. · 2019-12-12T21:51:19.449Z · score: 30 (13 votes) · EA · GW

Thanks for the comment!

I just wanted to add my 2 cents. I work at FHI (near GPI) and am separately involved with Rethink Charity, so am involved (somewhat) in both.

I'd agree that Rethink Priorities' work is formatted very differently than that of GPI, but am really not sure I'd say it's lower quality on average. I'd have to spend much more time investigating both to be more sure of either side being "higher in quality" to whatever that could be compared (perhaps, differences in the rates of under-inspection-errors).

My impression is that Rethink Priorities is attempting optimizing it's research for EA endeavors. So, if you would prefer them spend more time to change the formatting or similar, I'm sure they would be curious to know. I imagine they could shift more to produce LaTeX-type work, but that may take more time; but if this community would find it correspondingly more valuable, it could be worth it.

I'm also interested in this question for my own work and similar.

Comment by oagr on Shapley values: Better than counterfactuals · 2019-11-24T21:31:20.039Z · score: 3 (2 votes) · EA · GW

Nice post!

Quick thought on example 2:

I just wanted to point out that what is described with Newton and Leibniz is a very, very simplified example.

I imagine that really, Newton and Leibniz wouldn't be the only ones counted. With Shapley values, all of the other many people responsible for them doing that work and for propagating it would also have shared responsibility. Plus, all of the people who would have invented calculus had the two of them not invented it also would have had some part of the Shapley value.

The phrase "The Shapley assigns equal value to equivalent agents." is quite tricky here, as there's a very specific meaning to "equivalent agents" that probably won't be obvious to most readers at first.

Of course, much of this complexity also takes place with counterfactual value. (As in, Newton and Leibniz aren't counterfactually responsible for all of calculus, but rather some speedup and quality difference, in all likelihood).

Comment by oagr on Shapley values: Better than counterfactuals · 2019-11-24T21:22:44.307Z · score: 2 (1 votes) · EA · GW

Is it possible to use Banzhaf values for generic attribution questions outside of voting? If so, can you link to some posts/papers that describe how to use it in such cases. The first set of things that came up are all voting-related.

Comment by oagr on Summary of Core Feedback Collected by CEA in Spring/Summer 2019 · 2019-11-07T18:45:03.232Z · score: 37 (18 votes) · EA · GW

Also happy to see this and the mistakes page (which I just realized existed). CEA has a pretty important but difficult position.

I would also be excited about this eventually getting more specific, though I realize that honesty does come with challenges. For instance, on the mistakes page, there's the shortcoming "We were too slow to distribute funds to student and local groups." This is obviously quite vague. It doesn't say when this happened or how big of an issue this was.

Also, one quick idea: I could imagine it may be worthwhile to hire external consultants or eventually organize a semi-extensive project to better understand what the experience of "joining the Effective Altruism movement" is like and trying to improve it accordingly. Service design, for instance, is used to understand how people go through complex experiences, like finding out about, traveling to, and experiencing Disneyland. Here's a page on it's use for UK government services. Perhaps similar could be done to analyze all of the pain points for possible new enthusiasts. I imagine there are a lot of pain points that may not be obvious, even to people experienced with things.

Comment by oagr on Introducing Foretold.io: A New Open-Source Prediction Registry · 2019-10-17T11:28:00.537Z · score: 2 (1 votes) · EA · GW

Thanks Soeren!

Comment by oagr on Introducing Foretold.io: A New Open-Source Prediction Registry · 2019-10-17T11:27:17.193Z · score: 6 (4 votes) · EA · GW

Thanks!

I believe the items in the "other useful features" section above are unique from Metaculus. Also, I've written this comment on the LessWrong post discussing things further.

https://www.lesswrong.com/posts/wCwii4QMA79GmyKz5/introducing-foretold-io-a-new-open-source-prediction?commentId=i3rQGkjt5CgijY4ow

Comment by oagr on [Link] "How feasible is long-range forecasting?" (Open Phil) · 2019-10-14T23:35:08.362Z · score: 4 (2 votes) · EA · GW

If forecasters are giving forecasts for similar things over different times, their resolution should very obviously decrease with time. A good example of this are time series forecasts, which grow in uncertainty over time projected into the future.

bicycle-projection

To site my other comment here, the tricky part, from what I could tell is calibration, but this is a more narrow problem. More work could definitely be done to test calibration over forecast time. My impression is that it doesn't fall dramatically, probably not enough to make a very smooth curve. I feel like if it were the case that it reliably fell for some forecasters, and those forecasters learned that, they could adjust accordingly. Of course, if the only feedback cycles are 10-year forecasts, that could take a while.

Image from the Bayesian Biologist: https://bayesianbiologist.com/2013/08/20/time-series-forecasting-bike-accidents/

Comment by oagr on [Link] "How feasible is long-range forecasting?" (Open Phil) · 2019-10-14T23:27:07.878Z · score: 14 (5 votes) · EA · GW

Happy to see this focus. I still find it quite strange out how little attention the general issue has gotten from other groups and how few decent studies exist.

I feel like one significant distinction for these discussions is that of calibration vs. resolution. This was mentioned in the footnotes (with a useful table) but I think it may deserve more attention here.

If long-term calibration is expected to be reasonable, then I would assume we could get much of the important information we could be interested in about forecasting ability from the resolution numbers. If forecasters are confident in predictions for a 5-20+ year time frame, this would be evident in corresponding high-resolution forecasts. If we want to compare these to baselines we could set them up now and compare resolution numbers.

We could also have forecasters do meta-forecasts; forecasts about forecasts. I believe that the straightforward resolution numbers should provide the main important data, but there could be other things you may be interested. For example, "What average level of resolution could we get on this set of questions if we were to spend X resources forecasting them?" If the forecasters were decently calibrated the main way this could go poorly is if the predictions to these questions would be low resolution, but if so that would be apparent quickly.

The much trickier thing seems to be calibration. If we cannot trust our forecasts to be calibrated over long time horizons, then the resolution of their forecasts is likely to be misleading, possibly in a highly systematic and deceiving way.

However, long-term calibration seems like a relatively constrained question to me, and one with possibly a pretty positive outlook. My impression from the table and spreadsheet is that in general, calibration was shown to be quite similar for short and long term forecasts. Also, it's not clear to me why calibration would be dramatically worse in long-term questions than it would be in specific short-term questions that we could test for cheap. For instance, if we expected that forecasters may be poorly calibrated on long-term questions because the incentives are poor, we could try having forecasters forecast very short-term questions with similarly poor incentives. I recall reading Anthony Aguirre speculating that he didn't expect Metaculus's forecaster's incentives to change much for long-term questions, but I forgot where this was mentioned (it may have been a podcast).

Having some long-term studies seems quite safe as well, but I'm not sure how much extra benefit they will give us compared to more rapid short-term studies combined with large sets of long-term predictions by calibrated forecasters (which should come with numbers of resolution).

Separately, I missed the footnotes on my first read through, but think that may have been my favorite part of it. The link is a bit small (though clicking on the citation numbers brings it up).

Comment by oagr on Leverage Research: reviewing the basic facts · 2019-10-09T16:24:32.002Z · score: 13 (5 votes) · EA · GW

Yep, understood, and thanks for clarifying in the above comment. I wasn't thinking you thought many of them were racist, but did think that at least a few readers may have gotten that impression from the piece.

There isn't too much public discussion on this topic and some people have pretty strong feelings on Leverage, so sadly sometimes the wording and details matter more than they probably should.