Introducing the ML Safety Scholars Program

post by ThomasW (ThomasWoodside), Dan Hendrycks, Mantas Mazeika, Oliver Zhang, Sidney Hough (Sidney), Kevin Liu (kliu128) 路 2022-05-04T13:14:07.422Z 路 EAGW38 comments


  Program Overview
  Why have this program?
  Time Commitment
  Preliminary Content & Schedule
    ML Safety
  Who is eligible?

Program Overview

The Machine Learning Safety Scholars program is a paid, 9-week summer program designed to help undergraduate students gain skills in machine learning with the aim of using those skills for empirical AI safety research in the future. Apply for the program here by May 31st.

The course will have three main parts:

The first two sections are based on public materials, and we plan to make the ML safety course publicly available soon as well. The purpose of this program is not to provide proprietary lessons but to better facilitate learning:

MLSS will be fully remote, so participants will be able to do it from wherever they鈥檙e located. 

Why have this program?

Much of AI safety research currently focuses on existing machine learning systems, so it鈥檚 necessary to understand the fundamentals of machine learning to be able to make contributions. While many students learn these fundamentals in their university courses, some might be interested in learning them on their own, perhaps because they have time over the summer or their university courses are badly timed. In addition, we don鈥檛 think that any university currently devotes multiple weeks to AI Safety.

There are already sources of funding for upskilling within EA, such as the Long Term Future Fund. Our program focuses specifically on ML and therefore we are able to provide a curriculum and support to Scholars in addition to funding, so they can focus on learning the content.

Our hope is that this program can contribute to producing knowledgeable and motivated undergraduates who can then use their skills to contribute to the most pressing research problems within AI safety.

Time Commitment

The program will last 9 weeks, beginning on Monday, June 20th, and ending on August 19th. We expect each week of the program to cover the equivalent of about 3 weeks of the university lectures we are drawing our curriculum from. As a result, the program will likely take roughly 30-40 hours per week, depending on speed and prior knowledge.

Preliminary Content & Schedule

Machine Learning (content from the MIT open course)

Week 1 - Basics, Perceptrons, Features

Week 2 - Features continued, Margin Maximization (logistic regression and gradient descent), Regression

Deep Learning (content from a University of Michigan course as well as an NYU course)

Week 3 - Introduction, Image Classification, Linear Classifiers, Optimization, Neural Networks. ML Assignments due.

Week 4 - Backpropagation, CNNs, CNN Architectures, Hardware and Software, Training Neural Nets I & II. DL Assignment 1 due.

Week 5 - RNNs, Attention, NLP (from NYU), Hugging Face tutorial (parts 1-3),

RL overview. DL Assignment 2 due.

ML Safety

Week 6 - Risk Management Background (e.g., accident models), Robustness (e.g., optimization pressure). DL Assignment 3 due.

Week 7 - Monitoring (e.g., emergent capabilities), Alignment (e.g., honesty). Project proposal due.

Week 8 - Systemic Safety (e.g., improved epistemics), Additional X-Risk Discussion (e.g., deceptive alignment). All ML Safety assignments due.

Week 9 - Final Project (edit May 5th: If students have a conflict in the last week of the program, they can choose not to complete the final project. Students who do this will receive a stipend of $4000 rather than $4500.)

Who is eligible?

The program is designed for motivated undergraduates who have interest in doing empirical AI safety research in the future. We will accept Scholars who will be enrolled undergraduate students after the conclusion of the program (this includes graduated/soon graduating high school students about to enroll in their first year of undergrad).


We don鈥檛 assume any ML knowledge, though we expect that the course could be helpful even for people who have some knowledge of ML already (e.g., or Andrew Ng鈥檚 Coursera course).


Questions about the program should be posted as comments on this post. If the question is only relevant to you, it can be addressed to Thomas Woodside ([firstname].[lastname]


We would like to thank the FTX Future Fund regranting program for providing the funding for the program.


You can apply for the program here. Admission is rolling, but you must apply by May 31st to be considered for the program. All decisions will be released by June 7th.


Comments sorted by top scores.

comment by electroswing2022-05-04T21:49:44.507ZEA(p) 路 GW(p)

You may already have this in mind but鈥攊f you are re-running this program in summer 2023, I think it would be a good idea to announce this further in advance [EAGW].

Replies from: ThomasWoodside
comment by ThomasW (ThomasWoodside)2022-05-05T01:49:11.473ZEA(p) 路 GW(p)

I completely agree! Summer plans are often solidified quite early, so promoting earlier is better. I'm no stranger to the idea of doing things early [EAGW]!

In this case, we saw a need the need for this program only a few weeks ago and we're now trying to fill it. If we do run it again next year, we'll announce it earlier, though there's definitely still some benefit to having applications open fairly late (e.g. for people who may not have gotten other positions because they lacked ML knowledge).

comment by Ethan (EJ) Watkins2022-05-13T03:10:12.953ZEA(p) 路 GW(p)

Regarding the prerequisites (differential calculus, linear algebra or statistics, programming) do you have any recommended resources for people who have studied these but are a bit rusty and want to revise the content before MLSS? Or a list of concepts or tasks that participants can use to check whether their understanding is at the required level? (e.g. "If you know the concepts X, Y and Z in linear algebra, and can write a program to solve task A and B, then you probably have enough background for MLSS.")

Replies from: Ethan (EJ) Watkins
comment by Ethan (EJ) Watkins2022-05-24T06:54:01.430ZEA(p) 路 GW(p)

Document provided by MLSS:
Preparation Materials and Curriculum [public]

Replies from: ThomasWoodside
comment by ThomasW (ThomasWoodside)2022-05-24T07:04:52.722ZEA(p) 路 GW(p)

Sorry, missed replying to this comment as we were working on this doc, this is indeed the resource we recommend!

comment by electroswing2022-05-06T22:51:57.734ZEA(p) 路 GW(p)

Can undergraduates who already know ML skip weeks 1-2? Can undergraduates who already know DL skip weeks 3-5?

Replies from: ThomasWoodside
comment by ThomasW (ThomasWoodside)2022-05-07T04:07:14.084ZEA(p) 路 GW(p)

We'll consider this if there's enough demand for it! But especially for the latter option, it might make sense for students to work through the last three weeks on their own (ML Safety lectures will be public by then).

comment by Terry_Kim2022-05-06T05:45:08.024ZEA(p) 路 GW(p)

Besides the designated work hours, will the program mostly be synchronous? I am heavily interested in participating, but will be based outside the U.S. for the summer.

Thanks so much for your time!

Replies from: ThomasWoodside
comment by ThomasW (ThomasWoodside)2022-05-06T12:17:11.623ZEA(p) 路 GW(p)

It will be mostly asynchronous, with a few hours of synchronous content per week. We also expect to have sections at different times for people in different timezones so there should be one that works for you.

comment by Jieru Liao2022-06-06T08:08:06.549ZEA(p) 路 GW(p)

I'm wondering whether  all  admission notices been sent

Replies from: ThomasWoodside
comment by ThomasW (ThomasWoodside)2022-06-06T15:26:35.982ZEA(p) 路 GW(p)

No, we're still working on it! All decisions will be sent by tomorrow, June 7th, as indicated in this post.

Replies from: Jieru Liao
comment by Jieru Liao2022-06-13T17:31:51.873ZEA(p) 路 GW(p)

Hello! I have looked through this website for thousand times, but I still regret to find that I do not seem to be accepted into this project, but I still want to learn about the courses. In addition to the links that have been posted for most courses of the project, I would like to ask whether you will establish an open community to share homework content during the project, and the final project and ML Safety course in the third week, I would like to learn these, thank you!

Replies from: ThomasWoodside
comment by ThomasW (ThomasWoodside)2022-06-13T17:47:31.209ZEA(p) 路 GW(p)

This document will include all of that information (some of it isn't ready yet).

comment by Hui Wen2022-05-17T22:35:23.135ZEA(p) 路 GW(p)

May I know how many places are there available for this program? 

Replies from: ThomasWoodside
comment by ThomasW (ThomasWoodside)2022-05-18T02:10:54.124ZEA(p) 路 GW(p)

This will depend on the number of TAs we can recruit, our yield rate, and other variables so I can't give a good figure on this right now, sorry.

comment by L4Z3R (ronit2k2)2022-05-11T09:46:03.714ZEA(p) 路 GW(p)

It'd be really great if there were programs like these for Information Security / Cybersecurity fields as well :(

comment by Raymond D (ulna101)2022-05-04T23:23:54.497ZEA(p) 路 GW(p)

Where do you plan to publicize the ML safety course? I have a lot of interest  in this topic area but cannot  commit to  the full program.

Replies from: ThomasWoodside
comment by ThomasW (ThomasWoodside)2022-05-05T01:40:34.702ZEA(p) 路 GW(p)

We'll have a website, and probably also make a forum post here when we release it.

comment by Lorenzo Venieri2022-05-04T17:35:09.879ZEA(p) 路 GW(p)

Is there any chance to participate even if I got my bachelor's degree last year? I am in my first year as a grad student and I am extremely interested in these topics.

If not, is there any way to access the ML safety part material?  I feel the need for structured material to study from about this topic and I would gladly look at it also by myself. 

Best regards

Replies from: ThomasWoodside
comment by ThomasW (ThomasWoodside)2022-05-04T17:52:09.600ZEA(p) 路 GW(p)

We're prioritizing undergraduates, but depending on the strength/size of our undergraduate applicant pool, we may also admit graduate students. Feel free to apply!

The ML Safety curriculum is not yet fully ready, but when it is we will release it publicly, not just for this program. We'll post again when we do.

Replies from: sreers, Lorenzo Venieri
comment by sreers2022-05-13T02:51:31.891ZEA(p) 路 GW(p)

Relatedly, is there any chance you'd consider people who have recently completed undergrad, but aren't grad students (e.g. are currently working in the corporate sector)?

Replies from: ThomasWoodside
comment by ThomasW (ThomasWoodside)2022-05-13T18:47:17.813ZEA(p) 路 GW(p)

We may consider people in this situation, but it's not the focus of our program and we will prioritize undergraduates.

comment by Lorenzo Venieri2022-05-05T07:29:14.764ZEA(p) 路 GW(p)

This is great news! I'll apply soon. Thanks

comment by J.Lim (Lim)2022-05-16T23:15:31.183ZEA(p) 路 GW(p)

After submitting my application, should I be getting a form confirmation email? Or is it fine not to have received anything? I wasn't sure if my application was processed correctly. Thanks!

Replies from: ThomasWoodside
comment by ThomasW (ThomasWoodside)2022-05-17T02:25:43.753ZEA(p) 路 GW(p)

A confirmation email is not expected. We received your application!

comment by Lexley Villasis (lei2gh)2022-08-19T12:54:21.923ZEA(p) 路 GW(p)

Hi, sadly just saw this post now but I'd definitely apply if you'll be hosting this program again next summer! Is there any way to get notified in case?

comment by Dhruv Channa2022-06-01T19:05:41.885ZEA(p) 路 GW(p)

I submitted my application yesterday, but haven鈥檛 received any confirmation email or anything of the like. Are we to expect something to confirm our application has been received, or is there any way I can make sure that all my materials have been submitted. Thanks

Replies from: Lily Ren
comment by Lily Ren2022-06-04T00:33:59.005ZEA(p) 路 GW(p)

Same question here.

comment by rich.scout2022-05-27T14:39:13.158ZEA(p) 路 GW(p)

Does the time "30-40 hours/week " refer to the equivalent time of lectures per week or the estimated total studying time (that is, the lectures + the time we use to do the assignment and revise the lessons) we may devote to the program every week? 

Replies from: ThomasWoodside
comment by ThomasW (ThomasWoodside)2022-05-27T16:14:41.572ZEA(p) 路 GW(p)

Total time including assignments. Don't worry, there will not be 30-40 hours of lecture videos every week!

comment by adamaria2022-05-24T19:03:41.972ZEA(p) 路 GW(p)

Are there select hours that we must be available? I have commitments at certain times of the day, and timezones may not match up. Would this be a problem?

Replies from: ThomasWoodside
comment by ThomasW (ThomasWoodside)2022-05-24T20:50:25.401ZEA(p) 路 GW(p)

That shouldn't be a problem. For synchronous activities, we will have multiple sessions you can attend (we will have people from all over the world so we need to do this anyway).

comment by rich.scout2022-05-20T12:56:48.641ZEA(p) 路 GW(p)

Am I able to get a reference letter from one of the professors or add this experience to my resume if I complete the program?

Replies from: ThomasWoodside
comment by ThomasW (ThomasWoodside)2022-05-20T14:30:41.812ZEA(p) 路 GW(p)

You can certainly add it to your resume, but you wouldn't be able to get a reference letter.

The program uses public recorded online classes, and while we have TAs, none of them are professors.

comment by Robi Rahman (robirahman)2022-05-20T00:42:06.612ZEA(p) 路 GW(p)

Someone referred me to apply to be a TA for this program. How would you like such people to contact you - should I email you, or is there another form for that?

Replies from: ThomasWoodside
comment by ThomasW (ThomasWoodside)2022-05-20T06:54:27.125ZEA(p) 路 GW(p)

Not clear right now whether we will need more TAs, but if we do, we'll make a post soon with an application. I'll reply to this if/when that happens. Thanks for your interest!

comment by J. Qiu2022-05-16T17:55:17.578ZEA(p) 路 GW(p)

Could we still apply if we have exams between Jun 20-Jun 24 and will be available afterwards?

Replies from: ThomasWoodside
comment by ThomasW (ThomasWoodside)2022-05-16T19:10:29.524ZEA(p) 路 GW(p)

Yes, but please note this on your application. In general, short periods of unavailability are fine, but we won't give any extensions for them so you will likely have to complete the material at an accelerated pace at the times when you are available.