EA Forum Reader

The 'freedom to trade internationally' composite, which is basically not-being-protectionist, saw one of the largest improvements of any of their scores over this time. They are still protectionist relative to many richer countries, like the JV requirements. But they are dramatically less protectionist than they used to be, and this change coincided with / preceded their dramatic growth.

Thank you for your comment, great to hear that these suggestions seem to be helpful.

And great questions. I don't have a definitive answer (I might miss some things and the situation can differ a lot between organisations) but I can share some thoughts. Your second question is easier to answer, so I will start with that one.

I think that it's a bad idea to work with volunteers if:

• The organisation does not realise that volunteers aren't free [EA · GW]. They cost time, for coordination, recruitment, training, answering questions, etc. And sometimes the organisation needs to pay for an insurance, the volunteer's equipment, etc.
• The organisation is not willing to invest in the relationship with the volunteer. Even though some volunteers work independently, everybody is still human and would like to have some relation to the organisation. (Or to say it more bluntly: if you are a jerk, don't work with volunteers (or other people))
• The organisation has the resources to pay people for their work. I think that, in general, it is preferable to compensate people for their work. Payment is a strong sign of value and appreciation, and if an organisation has the means to pay for the work, but doesn't do it, I would question what the additional value of the volunteer's work is. Besides being a great motivation killer for the volunteer, it seems to be a better use of the volunteer's time to volunteer somewhere else.

Ok, so when do organisations benefit the most from working with volunteers? I think this question is more complex than the former and depends on several factors. In general I think two components are important: 1. the nature of the work and 2. the available volunteers. If the organisation's work is not too specialised and there are enough people who fit with your organisation (and are willing to donate their time), then an organisation can benefit from working with volunteers. Otherwise, I think an organisation will struggle and it can become (very) hard to have an impact while working with volunteers.

1. The nature of the work.

Usually volunteers can be very skilled, but they do not have much time to become specialised. There are of course exceptions, but generally volunteers can not spend too much time on their work, therefore getting to know the specific organisation (and the specific work) less well than a staff member. So if your organisation's work needs a lot of inside knowledge, or is specific, it will be harder to work with volunteers. Of course, volunteers can be trained, but as they tend to spend less time on the job (and the turn-over is sometimes high), it can make the volunteer work so costly that it's not efficient anymore (for these costs, the organisation can better hire 1 person full-time and train him/her/them). Another factor that plays a role is how easily the work can be coordinated and managed, as all these are costs as well.

As a rule of thumb I think for short-term volunteering the work should be easy and straight forward (if you give somebody a description of the work, they must be able to carry it out without further instructions). For tasks that take longer, the work should contribute towards volunteer satisfaction. That is easier if the work can be carried out autonomous, is flexible, the volunteer is responsible for the whole process, and the work is challenging. If the organisation's work does not have any of these characteristics I assume it could be hard to retain your volunteers.

2. The available volunteers.

Your question was about when organisations benefit most from working with volunteers, but I think that this question can not be separated from the volunteers themselves. Partly because volunteers have to fit with your organisation, partly because volunteers who donate their time to one organisation, can not do that for another organisation (what is the best way to distribute this resource in the community?), and partly because if there are no qualified people to volunteer, the organisation can not have their intended impact. And as an organisation usually needs more volunteers to carry out the same work as one paid employee (because of the time availability), and it can be hard to find reliable, high-quality volunteers, an organisation can be easily bottlenecked by a lack of volunteers (some people say that good volunteers are power-law distributed).

Another factor is if the volunteering is the best use of the volunteer's time. Somebody could not only donate their time to another (more effective?) organisation, but also use their time to e.g. skill-up or earn to give. For effective volunteering that means that volunteers often volunteer to learn something, improve a skill, or test out a career option. If your organisation can offer an opportunity for people to do one of these, it could be encouraged (and positive for the community) to create volunteer opportunities. But if the organisation's work provides an opportunity that people are not looking for, it can be really hard to fill your positions.

I think his point was related to development via protection, etc., that was then loosened somewhat. But not sure.

Hickel claims that China's very non-neoliberal policy enriched its people

China's post-1979 reforms are one of the textbook examples of neoliberalism! They privatized many businesses, allowed the creation of markets for many goods and services, opened up to international trade and reduced capital controls. While there is still a great deal of central control, the level is dramatically lower than it was in the 1970s. Their economic freedom ranking improved from a terrible 3.59 in 1980 to a respectable (though still not great) 6 in 2002, a very rapid rise. This is a similar increase to other countries undergoing neoliberal reforms at the same time, like the UK, Chile and Sweden, though many of these started from a higher base.

EDIT: Unless you are talking about the One Child Policy, which I would agree is very non-neoliberal, and is a major policy.

This sounds like a good battle plan! And do let us know what you finally decide on!

Oh, also I want to be clear that what I was concerned about is the broader structure of argument, rather than being critical of your post! :)

Sorry if this isn’t as polished as I’d hoped. Still a lot to read and think about, but posting as I won’t have time now to elaborate further before the weekend. Thanks for doing the AMA!

It seems like a crux that you have identified is how “sudden emergence” happens. How would a recursive self-improvement feedback loop start? Increasing optimisation capacity is a convergent instrumental goal. But how exactly is that goal reached? To give the most pertinent example - what would the nuts and bolts of it be for it happening in an ML system? It’s possible to imagine a sufficiently large pile of linear algebra enabling recursive chain reactions of both improvement in algorithmic efficiency, and size (e.g. capturing all global compute -> nanotech -> converting Earth to Computronium). Even more so since GPT-3. But what would the trigger be for setting it off?

Does the above summary of my take of this chime with yours? Do you (or anyone else reading) know of any attempts at articulating such a “nuts-and-bolts” explanation of “sudden emergence” of AGI in an ML system?

Or maybe there would be no trigger? Maybe a great many arbitrary goals would lead to sufficiently large ML systems brute-force stumbling upon recursive self-improvement as an instrumental goal (or mesa-optimisation)?

Responding to some quotes from the 80,000 Hours podcast:

“It’s not really that’s surprising, I don’t have this wild destructive preference about how they’re arranged. Let’s say the atoms in this room. The general principle here is that if you want to try and predict what some future technology will look like, maybe there is some predictive power you get from thinking about X percent of the ways of doing this involve property P. But it’s important to think about where there’s a process by which this technology or artifact will emerge. Is that the sort of process that will be differentially attracted to things which are let’s say benign? If so, then maybe that outweighs the fact that most possible designs are not benign.”

What mechanism makes AI be attracted to benign things? Surely only through human direction? But to my mind the whole Bostrom/Yudkowsky argument is that it FOOMs out of control of humans (and e.g. converts everything into Computronium as a convergent instrumental goal.)

“There’s some intuition of just the gap between something that’s going around and let’s say murdering people and using their atoms for engineering projects and something that’s doing whatever it is you want it to be doing seems relatively large.”

This reads like a bit of a strawman. My intuition for the problem of instrumental convergence is that in many take-off scenarios the AI will perform (a lot) more compute, and the way it will do this is by converting all available matter to Computronium (with human-existential collateral damage). From what I’ve read, you don’t directly touch on such scenarios. Would be interested to hear your thoughts on them.

“my impression is that you typically won’t get behaviours which are radically different or that seem like the system’s going for something completely different.”

Whilst you might not typically get radically different behaviours, in the cases where ML systems do fail, they tend to fail catastrophically (in ways that a human never would)! This also fits in with the notion of hidden proxy goals from “mesa optimisers” being a major concern (as well as accurate and sufficient specification of human goals).

A mixture of conversations and shared Google Docs. Nothing publicly citable as far as I know.

Thanks, I'll update the text when I get access to Metaculus again (I've blocked myself from it for productivity reasons lol)

Interesting article. I would like to raise one quabble:

Advocates for traditional diversity metrics such as race, gender and class do so precisely because they track different ways of thinking.

I agree this is the stated reason for many corporate diversity advocates, but I think it is not their true reason. In practice many companies recruit using basically a combination of filters whose purpose is to select people with a certain way of thinking (e.g. resumes, interviews, psychological screens) combined with various quotas for desired racial groups. If getting cognitive diversity was the goal they would try testing and selecting for that directly, or at least stop actively selecting against it. The status quo is likely to mean McKinsey get people from a variety of races, all of whom went to Harvard Business School, which I presume is basically what we want. After all, while cognitive diversity in some regards is useful, we want everyone to have the same (high) level of the cluster of skills that make up being a good consultant, like diligence, intelligence and sociability.

In particular, that even if hypothetically research showed that traditional racial/sexual diversity inhibited useful cognitive diversity (perhaps by making people less comfortable about sharing their views), advocates would be unlikely to change their mind.

I think their true motivations are more like some combination of:

• Desire to appeal to a variety of audiences who would be less likely to buy from an outsider (e.g. hiring black sales guys to sell in black areas).
• Wanting to avoid being criticized as racist by hostile outsiders.
• Left wing conceptions of fairness on behalf of HR staff and other management, unrelated to firm objectives.
• Intellectual conformism with others who believe for the previous three reasons.

A non-exhaustive subset of admired individuals I believe includes: ... As far as I perceive it, all revered individuals are male.

It seems a little rude to make public lists of perceived intelligence. Imagine how it would feel to be a prominent EA and to be excluded from the list? :-( In this case, I think you have excluded some people who are definitely higher in community estimation that some on your list, including some prominent women.

Members write articles [LW · GW] about him in apparent awe and possibly jest

The linked article is from over eleven years ago. I think GWWC hadn't even launched at that point, let alone the rest of the EA community. This is like attacking democrats because Obama thought gay marriage was immoral and was trying to build a border wall with Mexico, both of which were the case in 2009.

Phil Trammell makes a similar point about how we should update from the fact that smart people refuse to engage with AI risk arguments:

" The upshot here seems to be that when a lot of people disagree with the experts on some issue, one should often give a lot of weight to the popular disagreement, even when one is among the experts and the people's objections sound insane. Epistemic humility can demand more than deference in the face of peer disagreement: it can demand deference in the face of disagreement from one's epistemic inferiors, as long as they're numerous. They haven't engaged with the arguments, but there is information to be extracted from the very fact that they haven't bothered engaging with them. "