Comment by shaybenmoshe on Some promising career ideas beyond 80,000 Hours' priority paths · 2020-08-05T17:49:35.669Z · score: 5 (3 votes) · EA · GW

As someone in the intersection of these subjects I tend to agree with your conclusion, and with your next comment to Arden describing the design-implementation relationship.

However, while thinking about this, I did come up with a (very rough) idea for AI alignment , where formal verification could play a significant role.
One scenario for AGI takeoff, or for solving AI alignment, is to do it inductively - that is, each generation of agents designs the next generation, which should be more sophisticated (and hopefully still aligned). Perhaps one plan to do achieve this is as follows (I'm not claiming that any step is easy or even plausible):

  1. Formally define what it means for an agent to be aligned, in such a way that subsequent agents designed by this agent are also aligned.
  2. Build your first generation of AI agents (which should be lean and simple as possible, to make the next step easier).
  3. Let a (perhaps computer assisted) human prove that the first generation of AI is aligned in the formal sense of 1.

Then, once you deploy the first generation of agents, it is their job to formally prove that further agents designed by them are aligned as well. Hopefully, since they are very intelligent, and plausibly good at manipulating the previous formal proofs, they can find such proofs. Since the proof is formal, humans can trust and verify it (for example using traditional formal proof checkers), despite not being able to come up with the proof themselves.

This plan has many pitfalls (for example, each step may turn out to be extremely hard to carry out, or maybe your definition of alignment will be so strict that the agents won't be able to construct any new and interesting aligned agents), however it is a possible way to be certain about having aligned AI.

Comment by shaybenmoshe on Climate change donation recommendations · 2020-07-19T18:09:30.072Z · score: 1 (1 votes) · EA · GW

I agree with your main argument, but I think that the current situation is that we have no estimate at all, and this is bad. We literally have no idea if GFI averts 1 ton CO2e at $0.01 or at $1000. I believe having some very rough estimates could be very useful, and not that hard to do.

Also, I completely agree that splitting donations is a very good idea, and I personally do it (and in particular donated to both CATF and GFI in the past).

Comment by shaybenmoshe on Climate change donation recommendations · 2020-07-19T12:40:12.948Z · score: 1 (1 votes) · EA · GW

Thanks for sharing your perspective. However, I disagree with the conclusion of not performing these evaluations for that reason (though I think that it might make it harder to analyze and give an accurate answer).

For example, if it turns that GFI is 7 times less effective then CATF, that might mean that GFI is an extremely good donation opportunity for someone who wants to support both animal welfare and climate change mitigation. If it turns out that GFI's impact is 1000 times less effective then CATF, then the impact on climate change is negligible in donating to them.

Knowing the answer to this question could impact many people's donation strategy, especially if they are uncertain about what are the most important causes and prefer a diverse portfolio (like me).

Comment by shaybenmoshe on Climate change donation recommendations · 2020-07-19T11:34:27.758Z · score: 3 (2 votes) · EA · GW

Thank you for writing that up!

Do you (or anyone else) have any cost-effectiveness analysis of CO2e emissions averted (even if very rough) for the charities in appendix 2?
I am particularly interested in estimates for the Good Food Institute impact on CO2e emissions.

Comment by shaybenmoshe on I'm Michelle Hutchinson, head of advising at 80,000 Hours, AMA · 2020-04-29T22:02:04.556Z · score: 1 (1 votes) · EA · GW

Thanks for the follow up!

Comment by shaybenmoshe on Why I'm Not Vegan · 2020-04-11T07:51:50.078Z · score: 9 (5 votes) · EA · GW

I completely agree, and I too was troubled by this analysis. For me, the bottom line is:
The fact that something is of little-to-no cost, does not mean that its moral value is also little.

Furthermore, in cases like reducing animal suffering, one can both avoid being harmful himself (i.e. become vegan) AND donate to relevant charities, rather than OR.

Comment by shaybenmoshe on What should EAs interested in climate change do? · 2020-01-14T07:00:30.116Z · score: 2 (2 votes) · EA · GW

Good to know! Is there any information about Founders Pledge research project?

Comment by shaybenmoshe on I'm Michelle Hutchinson, head of advising at 80,000 Hours, AMA · 2019-12-08T12:35:32.829Z · score: 1 (1 votes) · EA · GW

Oh I see, I misunderstood you.

Thanks, looking forward to the episode to come out.

Comment by shaybenmoshe on I'm Michelle Hutchinson, head of advising at 80,000 Hours, AMA · 2019-12-07T09:44:24.783Z · score: 1 (1 votes) · EA · GW

Thank you for this answer (and the rest of them!). Could you link to that podcast episode on advising?

Comment by shaybenmoshe on [deleted post] 2019-08-26T16:57:57.181Z

First of all, I think it is a reasonable assumption that there are only finitely many people in the universe, in which case the order does not matter.

Also, of course the location of people matter. If you move all earth people at this moment to mars, we will all suffer and die. In the same way, say there 100 other galaxies in which there is another earth. Move all of these people here, and we will have an enormous over population.

Comment by shaybenmoshe on [deleted post] 2019-08-19T13:56:21.307Z

Hey, this might not really help, but here's a rough idea. I can think about it more thoroughly, or we can discuss this at some time, if you want.

Maybe you shouldn't consider the situation as two possibilities of a finite universe or infinite universe. An infinite universe is the limit of finite universes. Perhaps then you should consider an infinite universe as a sequence of finite universes, whose limit is indeed infinite, and work with that.

So for example, you will compute the difference in value in each of the finite universes, and take the limit of that, and use this as the value for the infinite universe.

This is a standard method in math and in physics. For example, in math this is formalized in pro categories (such as profinite groups), and in physics this is a sort of renormalization/regularization.

Comment by shaybenmoshe on What is the current best estimate of the cumulative elasticity of chicken? · 2019-05-04T20:40:43.655Z · score: 1 (1 votes) · EA · GW

Not exactly answering your question, but I think this argument (and follow up question) neglect an important aspect of your contribution.

Having more vegetarian and vegan people creates an incentive to develop meat substitutes (e.g. beyond meat, impossible).
If those substitutes, and especially clean meat, will hold up to their promise (i.e. cheaper, taste just as good, and be at least as healthy as regular meat), it will have the potential to change the meat industry dramatically.
In this situation, way more people may become vegetarian or vegan due to economical or health reasons.