The gap between prestige and impactpost by Adam Binks, ojorgensen · 2022-08-04T21:38:27.688Z · EA · GW · 1 comments
Examples of proxy goals that might be misaligned with big goals: What to do about this None 1 comment
Here’s a pattern that we noticed. You probably have some large goal you’re optimising for, like “learn to build great software”. But day-to-day, you optimise for a proxy goal, like “write code which my supervisor says is good”.
Proxy goals are often useful: they’re easier to evaluate, which makes it easier to make quick decisions.
For example, if your big goal is to write a bestselling novel, this is hard to evaluate - to find out if you’re achieving it, you might need to survey readers or even publish the novel. It’s easier to use the proxy of writing a novel that you think is great.
But sometimes, it’s unclear that your proxy tracks your big goal.
For example, if your big goal is to help solve the alignment problem, you might find yourself using the proxy goal of doing things that other people in the EA community think are prestigious.
However, there’s uncertainty here. Prestige within the community doesn’t always align with impact. Working on an established research agenda might be immediately impressive, while building your own research agenda might be confusing or illegible to other community members.
Examples of proxy goals that might be misaligned with big goals:
- Big goal: doing impactful research. Proxy goal: publishing papers in prestigious venues.
- Possible failure mode: doing research that is easier/faster to publish but less impactful.
- Big goal: provide for my family. Proxy goal: work really hard.
- Possible failure mode: working so hard you are not emotionally present for your family.
- Big goal: reduce the risk of a nuclear strike. Proxy goal: reach a powerful position in the US government.
- Possible failure mode: to climb the ranks, you take actions that have a negative impact, and it’s unclear if this negates your hypothetical future positive impact in expectation.
What to do about this
- Be aware that you are using proxy goals.
- Periodically step back to notice how your proxy goals misalign with your big goals.
- Re-engineer your proxy goals or incentive environment to make sure that the work you’re doing actually promotes your big goal.
Thanks to Isaac Dunn and Jarred Filmer for great conversations that led to this.
Comments sorted by top scores.