Does China have AI alignment resources/institutions? How can we prioritize creating more? 2022-08-04T19:23:24.175Z


Comment by jskatt ( on AI safety starter pack · 2022-08-11T04:05:56.282Z · EA · GW

This list is great. I recommend adding the new Intro to ML Safety course and the ML Safety Scholars Program. Or maybe everyone is supposed to read Charlie's post for the most up-to-date resources? It's worth clarifying.

Comment by jskatt ( on A list of EA-related podcasts · 2022-08-08T18:10:11.758Z · EA · GW

The Inside View also focuses on AI alignment. There's a YouTube channel with videos of the interviews. Sometimes there are interview highlights on LessWrong.

Comment by jskatt ( on EA in the mainstream media: if you're not at the table, you're on the menu · 2022-08-07T22:22:17.941Z · EA · GW

Here are some comments on the article that I sent to my family.

In 1972 philosopher Peter Singer suggested using metrics rather than emotion to direct charitable giving.

Not sure what he's talking about. I think the main point of Famine, Affluence, and Morality is that if you can help someone without a significant cost to yourself, you should.

Effective altruism also seems to be related to the “work to give” movement. Workers will rationalize high-paying jobs by giving most of their income away. Actually, when you work, you already give to society, but that is too complex for some to understand.

Earning to give is only a small part of EA, and I don't think it's typically a post hoc rationalization. And EAs understand very well that working directly on problems can give to society - see the first WSJ article I sent.

An organization known as GiveWell will tell you what charities are effective. I did a little digging, and I’m not so sure they’re effective at all. Yes, they direct money toward malaria nets and treatments for parasitic worms, but they also supply supplements for vitamin A deficiency, though genetically modified “golden” rice already provides vitamin A more effectively. Hmmm, seems like a move backward.

It's plausible that the best way to reduce vitamin A deficiency is to invest in multiple strategies at once. But if he gave a thorough argument that donating to "golden" rice infrastructure fights vitamin A deficiency more effectively per dollar than vitamin A supplementation, then I wouldn't be surprised to see GiveWell change its recommendations.

William MacAskill, a major effective-altruism booster, told the Washington Post that more should be spent on “preparing for low-probability, high-cost events such as pandemics.” That’s a bit like closing the barn door after the horse has bolted.

The author's comment seems quite silly to me.

And Mr. Bankman-Fried’s various entities, along with Cari Tuna and others, have put up about $19 million for a future California ballot measure, the California Pandemic Early Detection and Prevention Act, which would add a 0.75% tax on incomes over $5 million to raise up to $15 billion over 10 years. Catch that? Someone else pays. Effective, but not exactly selfless.

I don't see anything wrong with SBF promoting a tax on extremely wealthy people to prevent pandemics (unless the resulting pandemic prevention efforts are less valuable than what the wealthy people would do with their money otherwise). In general, I'm sure some taxes are totally worth promoting.

I don’t care if altruists spend their own money trying to prevent future risks from robot invasions or green nanotech goo, but they should stop asking American taxpayers to waste money on their quirky concerns.

Pandemic prevention is not a "quirky" concern!

And “effective” is in the eye of the beholder. Effective altruism proponent Steven Pinker said last year, “I don’t particularly think that combating artificial intelligence risk is an effective form of altruism.”

Yes, EAs don't agree on everything, nor do I think they should. There's an emphasis within EA on updating your beliefs in response to new evidence, such as reasonable arguments from other people.

Development economist Lant Pritchett finds it “puzzling that people’s [sic] whose private fortunes are generated by non-linearity”—Facebook, Google and FTX can write code that scales to billions of users—“waste their time debating the best (cost-effective) linear way to give away their private fortunes.”

So the argument is that when deciding where to donate your money, you should use the same tactics that earned you that money in the first place? It's unclear how "cost-effectiveness" is the same as "linearity." Maybe he's advocating for donating to interventions that are like unicorn startups - interventions that could be hugely beneficial if they succeed, but probably won't do much. If so, this is kind of exactly what Open Philanthropy is doing ("hits-based giving")

He notes that “national development” and “high economic productivity” drive human well-being. So true. History has proved that capitalism is the most effective and altruistic system.

It's fully possible to believe in EA principles and support capitalism. But high economic productivity can come with damaging externalities, such as increased risk of global catastrophes from new technologies.

There are only four things you can do with your money: spend it, pay taxes, give it away or invest it. Only the last drives productivity and helps society in the long term.

That seems totally incorrect. GiveWell estimates that donations to its recommended charities have averted over 100,000 deaths.

Eric Hoffer wrote in 1967 of the U.S.: “What starts out here as a mass movement ends up as a racket, a cult, or a corporation.” That’s true even of allegedly altruistic ones.

This is one of the few points in the article that I like. EA (which EA headquarters likes to describe as "a project") resembles a cult in some ways: people worry about future catastrophes, care about "doing good," think about weird ideas, and dream about growing the movement.

Comment by jskatt ( on Does China have AI alignment resources/institutions? How can we prioritize creating more? · 2022-08-07T20:55:00.634Z · EA · GW

Vael Gates's post "Resources I send to AI researchers about AI safety" offers this:

AI Safety in China

Comment by jskatt ( on What does ‘one dollar of value’ mean? · 2022-08-07T18:38:22.752Z · EA · GW

The Kendrick Lamar joke at the top makes me a little uncomfortable since that song (and more generally, that album) is about a very serious topic. Otherwise I really like this post; I'm also confused about the precise meaning of "doing X is worth $$$." 

Comment by jskatt ( on Most Ivy-smart students aren't at Ivy-tier schools · 2022-08-07T18:33:50.740Z · EA · GW

I think this post argues that people shouldn't obsess about elite universities as sources of talent. My paraphrasing of the title is "Most super smart students aren't at super elite schools."

Comment by jskatt ( on How long does it take to undersrand AI X-Risk from scratch so that I have a confident, clear mental model of it from first principles? · 2022-08-06T22:59:02.114Z · EA · GW

Here's the most up-to-date version of the AGI Safety Fundamentals curriculum. Be sure to check out Richard Ngo's "AGI safety from first principles" report. There's also a "Further resources" section at the bottom linking to pages like "Lots of links" from AI Safety Support.

Comment by jskatt ( on Why should someone do the MOST good? Does a normative argument even exist? · 2022-08-06T22:49:58.887Z · EA · GW

The ethical theory of utilitarianism essentially states that "we ought to act to improve the well-being of everyone by as much as possible," which has a strong "do the most good" vibe. There are certainly a lot of arguments for and against utilitarianism-style ethics.

I think one relevant intuition people have is "there's never a point at which I would not want to help any more people." Like if one action helps N people (in expectation) and another helps N+1 people, I'd rather do the latter.

A conflicting intuition is that we don't feel that much better about helping 10 billion people than helping 9 billion people. The essay "On caring" argues that it's still really important to help those extra 1 billion people.

Another idea is that not maximizing (e.g. not saving an extra person's life because you used that money to eat at a fancy restaurant) is the same as allowing harm to happen, and some philosophers believe that this is no different than doing harm.

You may also be interested in the Von Neumann–Morgenstern utility theorem, which proves that all agents whose behavior obeys some reasonable properties will behave as maximizers.

As a side note, I don't think you need to deeply care about maximization to care about EA; for example, you might feel fine about frequenting fancy restaurants. EA is not utilitarianism; the core idea of EA is increasing the quality of your altruism, not the quantity (although plenty of EAs feel inspired to increase the quantity as well, and some of these EAs are utilitarians).

Comment by jskatt ( on Some non-EAs worry about EA's effect on mental health · 2022-08-06T20:54:55.181Z · EA · GW

People act like the difficult problems in front of them are the reason for the low moods they are having.

Sometimes this is true! In which case I recommend contemplating "Detach the grim-o-meter."

Here's a separate error that I've made many times: People believe that their intellectual knowledge of the world's problems causes them to act a certain way, when in reality they act that way because of their mood.

Comment by jskatt ( on Does China have AI alignment resources/institutions? How can we prioritize creating more? · 2022-08-06T17:54:33.260Z · EA · GW

That's great. Seems that these days all the Alignment Newsletter translations go directly onto the English website.

Comment by jskatt ( on $20K in Bounties for AI Safety Public Materials · 2022-08-06T16:52:43.359Z · EA · GW

This is a great idea! For the future, maybe make the bounty a little higher? Writing a super amazing post for this contest could take >100 hours, and the contest offers a possibility of getting paid <$20 per hour for that. I suspect there could be some 80/20 rule at work for the impact of these posts, so super amazing posts are especially important to incentivize.

Comment by jskatt ( on Does China have AI alignment resources/institutions? How can we prioritize creating more? · 2022-08-05T18:28:05.418Z · EA · GW

Thanks for sharing these. The Chinese Association for AGI appears to focus on advancing AI capabilities rather than AI safety. I used Google Translate to translate the lead paragraph of the website's current opening page:

Notice of the 7th China General Artificial Intelligence Annual Conference

The China General Artificial Intelligence Annual Conference has been successfully held for six consecutive sessions. It is an annual event for Chinese general artificial intelligence enthusiasts, involving computer science, philosophy, logic, education, psychology, sociology, law, medicine and other disciplines. In order to better demonstrate and promote the research and application of general artificial intelligence, the 7th China General Artificial Intelligence Annual Conference in 2022 will be held at Northwest University for Nationalities in Lanzhou City.

Comment by jskatt ( on Does China have AI alignment resources/institutions? How can we prioritize creating more? · 2022-08-05T18:02:50.377Z · EA · GW

Excellent, I'm happy to see that! However, I'm concerned that the proposal focuses entirely on translating general EA concepts.

Publications we might start with include, the 80,000 Hours ‘key idea’ series, and Toby Ord’s The Precipice.

I think it is much higher priority (from the perspective of reducing AI x-risk) to translate AI alignment concepts, particularly the AGI Safety Fundamentals course material. It takes a lot of inferences to go from "I'm interested in doing good" to "I like EA ideas" to "I think AI alignment is important" to "I want to work on AI, where can I start?" And even if many Mandarin speakers reach that last point through a Mandarin translation of 80,000 Hours, they will currently find very few (if any?) structured opportunities to skill up for AI alignment.

Additionally, I don't think one needs to know about longtermism and QALYs and PlayPumps to recognize the importance of AI alignment work. Nor does one need to care about doing as much good as possible with their career. One only needs to grasp why AI might be extremely dangerous and why advanced capabilities might be coming soon.

One more point is that translating AI alignment resources may have lower risks than translating general EA content.

Comment by jskatt ( on Meditation course claims 65% enlightenment rate: my review · 2022-08-02T19:52:36.205Z · EA · GW

In 2019 I spent a few months doing 45-60 minutes per day and experienced dramatic improvements to my well-being, basically from "somewhat bad" to "quite good." I've been able to maintain somewhere between "somewhat good" and "very good" for most of the time since then with only 10-20 minutes per day, which is what I had been doing for ~1 year before the more intense practice. This trend could be regression to the mean since I've historically had pretty high well-being. It may also be that the more intense practice resolved a particular mental difficulty I had been experiencing for ~6 months.

Note that intense meditation comes with higher risks; see Willoughby Britton's work. I would not recommend trying 10-day retreats or the Finder's Course without researching these risks and considering your personal risk factors.

Comment by jskatt ( on Reducing nightmares as a cause area · 2022-07-23T18:39:39.291Z · EA · GW

Nitpicking, but don't you mean 1.5x as important? If we neglect 8 hours of sleep, there are 16 hours of daily experience. If we include 8 hours of sleep, there are 24 hours of daily experience. And 24 is 1.5x as large as 16.

Comment by jskatt ( on What is EA not? · 2022-07-23T17:35:44.785Z · EA · GW

I'm having trouble understanding the focus of this post. What are its central theses? What is the TLDR?

Comment by jskatt ( on When Utilitarianism is Actually ✨Fun✨ · 2022-07-17T19:44:52.243Z · EA · GW

You're hanging out with your utilitarian friends. Alice has a really nice looking muffin. By your best objective estimation, you would enjoy the muffin more. Alice now has to give you the muffin or be sent to 👮 utilitarian jail 👮.

This made me laugh. Poor Alice. She's heading straight for👮 utilitarian jail 👮.

Comment by jskatt ( on Does the idea of AGI that benevolently control us appeal to EA folks? · 2022-07-17T19:35:34.263Z · EA · GW

Seriously, though, if the AGI really controlled us, it could decide what we wanted ...  Then it would do whatever we want. 

If we develop extremely capable and aligned AI, it might be able to form a model of any person's mind and give that person exactly what they want. But I think there will be a lot of intermediate AI systems before we get to that point. And these models will still be very capable, so we will still need them to be aligned, and we won't be able to achieve this by simply saying "model human minds and give us what we want."

Comment by jskatt ( on Why did I misunderstand utilitarianism so badly? · 2022-07-16T20:37:15.155Z · EA · GW

The first mistake has to do with an attitude towards learning. I used to not mind feeling burnt out and intellectually drained and so I'd continue studying even when I was at a state where I couldn't have the minimum of clarity needed to genuinely think and reflect on a topic. ... This is relevant because if you'd like to be good let's say at philosophical reasoning, you should be able to charitably understand and interpret the positions you're engaging with before taking sides.

This is a great point. In my experience, if I sit down and spend a long time thinking, I eventually stop thinking critically. I can still absorb new facts and ideas, but I lose most of my skepticism.

Comment by jskatt ( on Does the idea of AGI that benevolently control us appeal to EA folks? · 2022-07-16T20:22:00.346Z · EA · GW

An actual utopia sounds pretty good to me, but I don't think this vision is a solution to the alignment problem. It is something we might want an AGI to do for humanity, but we don't know how to ensure that an AGI does what we want.

Comment by jskatt ( on Aiming for the minimum of self-care is dangerous · 2022-07-16T19:47:38.534Z · EA · GW

But what’s often lost amongst depressed altruists is another core principle: some good things you can do matter 100x or 1000x more than others.

If you’re doing impact-oriented research, then picking an important thesis topic matters far more than whether you finish writing it in four years or five. If you’re earning to give, negotiating your salary matters far more than whether you take an additional two weeks of holiday. If you’re building community, following up with promising people matters far more than whether you postpone an event until next month.

This point is similar to Ramit Sethi's concept of "big wins" in personal finance. I have found this to be an extremely valuable heuristic.

You don't have enough time and energy to do everything perfectly. Some actions/decisions are much more important than others ("big wins"), so focus on doing those well. Don't waste time on trivialities.

There's a corollary that should be intuitive to the EA community. Someone working 40 hours per week can totally dwarf the altruistic output of someone working 60 hours per week.

Maintaining good mental health is a big win.

Comment by jskatt ( on On how various plans miss the hard bits of the alignment challenge · 2022-07-12T19:39:13.279Z · EA · GW

Maybe I'm missing something, but it seems that "dignity" only appears once in the OP? Namely, here:

On my model, solutions to how capabilities generalize further than alignment are necessary but not sufficient. There is dignity in attacking a variety of other real problems, and I endorse that practice.

This usage appears to have nothing to do with the April Fool's Day post.

Perhaps Soares made a subsequent edit to the OP?

Comment by jskatt ( on My Most Likely Reason to Die Young is AI X-Risk · 2022-07-07T17:12:38.282Z · EA · GW

"AGI risk" and "AGI safety" also seem good. But they don't immediately convey the high stakes.

Comment by jskatt ( on How To Prevent EA From Ever Turning Into a Cult · 2022-07-04T04:20:31.337Z · EA · GW

Frugality needs to stay a virtue in the EA culture to not attract members who are motivated primarily by money. Not that we should be offering sleeping bags and tents on our retreats – the ”EA standard” needs to be comfortable enough to be considered by individuals of the general public. But grants should not be used to pay for excesses for a selected few (ie. fancy hotels, resorts, or domestic work).

I agree that all-inclusive resorts are unnecessary and excessive in almost all situations. Not sure what "domestic work" means. However, I worry that a heavy emphasis on frugality could promote feelings of scarcity in some EAs and thereby cause suboptimal decisions. It's hard for people to do high quality work when they worry obsessively about money.

Comment by jskatt ( on A summary of every Replacing Guilt post · 2022-07-04T04:06:08.046Z · EA · GW

When we use the word “should”, we make one choice seem like the option that makes us a good person and one option seem like negative infinity.

I really like this sentence. I think it applies in many other scenarios beyond using "should." I have experienced plenty of unproductive perfectionism arising from a mistaken belief that the perfect option is good and everything else is terrible.

Comment by jskatt ( on EA Dedicates · 2022-06-26T19:05:39.848Z · EA · GW

I like most of this article.

There’s a sense of meaning and purpose to devoting yourself to world improvement: you never have to worry about ennui, about the sense that you’ll die without having any sort of impact on the world, that there’s no reason you’re alive.

I dislike this point, perhaps because it faintly promotes using utilitarianism to suppress normal human feelings. I think a healthier approach is to accept these feelings without obsessing over them. Additionally, the thought "Well, these are problems for other people, but not for me thanks to being a dedicate!" reminds me of blind faith, escapism, avoiding the least convenient possible world, and once-and-for-allism.

Dedicates don’t have bullshit jobs.

I think the same can be said for many non-dedicates.