A Viral License for AI Safety

post by IvanVendrov · 2021-06-05T02:00:26.557Z · EA · GW · 6 comments


  A Viral License for AI Safety: The GPL as a Model for Cooperation
    Potential License Clauses
    to broad principles
    and corrigibility
    license violations
    violation decisions
    for Future Work

A Viral License for AI Safety: The GPL as a Model for Cooperation

Written by Ivan Vendrov & Nat Kozak


The GPL or General Public License was originally written by Richard Stallman for the GNU Project. If someone publishes a codebase under the GPL, the license guarantees the users’ freedom to run it, study it, change it, and redistribute copies. It also requires that anyone distributing code that includes any part of the publisher’s code must also make their codebase comply with the GPL. This means that a for-profit company cannot distribute proprietary software with any part of the codebase that uses the GPL license. In contrast, permissive licenses like MIT or Apache do not have this restriction, so a for-profit company can use part of it inside their own proprietary codebase with few restrictions or consequences.

An interesting and important feature of the GPL is that it spreads virally: it attaches the “free software” property to any part of the constituent code, no matter how many copying steps exist between the developer who wants to use the code and the original codebase.

Instead of using the property “free software”, could we define a property we care about like “beneficial” or “safe” or at least “audited by an AI safety org” that likewise perpetuates itself? Further, how can we make sure that the codebase that defines the first AGI system satisfies this property?

Currently AI research is extremely cumulative. Most research does not start from scratch but builds on existing code bases and libraries, increasingly even pre-trained models. Almost all AI codebases and models are released under permissive licenses like MIT. If most research was circulated with a viral license such that actors could only use code or models if they committed to a set of practices and processes ensuring safety and alignment, it seems likelier that the first AGI system would follow these practices.

Some Potential License Clauses

Commitments to broad principles

There’s a class of clauses that constitutes a commitment to broad principles. Those principles could include things like:

Notable examples of such commitments in the context of AI systems include OpenAI's charter, Google’s 7 principles for responsible AI, and Partnership on AI’s 8 tenets.

Promising adherence to principles is easy. However, it doesn’t guarantee that they will be followed in any meaningful way. Environmental treaties, for instance, seem to have been most effective when specific milestones are articulated in addition to general principles.

On the other hand, even nonbinding commitments have historically changed the behavior of state actors. Take the result of the Helsinki Accords: a signed agreement that never had treaty status nonetheless successfully cooled border tensions and granted legitimacy to Soviet dissident and liberal movements.

Overall, it seems that an effective license should contain both commitments to broad principles and specific enforceable clauses, with clear delineations between the two.

Versioning and corrigibility

The GPL has an "or any later version" clause that optionally allows later changes to the license to override earlier changes. An AI license may benefit from a clause like this, or an even stricter clause that requires code licensed under any version to be enforceable under the latest version.

Such a clause may prove critical because it is likely that many important safety measures have not yet been invented. Likewise, many relevant values have not yet been discovered or articulated. However, such a restrictive versioning clause would probably adversely affect adoption and enforcement, potentially to an impractical degree.

Determining license violations

To start, we could look at the GPL enforcement mechanisms to see what they have tried, and what has and has not worked for the free software movement.

We could rely on a central authority such as a committee that includes respected members of the AI Safety and policy communities. This committee would have the authority to determine whether the license has or has not been broken. Instead of deciding each case manually, the committee could maintain an automated alignment benchmark against which new code is evaluated.

Legal systems could represent a final fallback. If your code has been used in a way that you think has violated the license, you could sue the violator. This would not deter actors outside the legal system’s reach, such as state actors.

Enforcing violation decisions

The standard legal remedies actually seem quite compelling. The history of GPL enforcement shows large companies being forced by courts around the world to comply with the license terms and pay significant damages for violations.

Besides legal remedies, community enforcement is possible. We can draw inspiration from international treaties that create extralegal enforceable commitments.

Just like with the usual justice system, there is a question of how someone might be able to come back, if ever, from violating it.

For a sufficiently drastic violation, the best available enforcement mechanism may be to entirely suspend publication of work building on licensed code and models. Potentially the license could include terms such that all participants must suspend publication.

Call for Future Work

An EA organization or group seeking to pursue this idea may want to:

Individuals could contribute in a number of ways, including

Thanks to Jonathan Rodriguez, Guive Assadi, Todor Markov, and Jeremy Nixon for helpful comments on earlier drafts of this article.


Comments sorted by top scores.

comment by technicalities · 2021-06-05T09:20:58.707Z · EA(p) · GW(p)

This is a neat idea, and unlike many safety policy ideas it has scaling built in.

However, I think the evidence from the original GPL suggests that this wouldn't work. Large companies are extremely careful to just not use GPL software, and this includes just making their own closed source implementations.* Things like the Skype case are the exception, which make other companies even more careful not to use GPL things. All of this has caused GPL licencing to fall massively in the last decade.** I can't find stats, but I predict that GPL projects will have much less usage and dev activity.

It's difficult to imagine software so good and difficult to replicate that Google would invite our virus into their proprietary repo. Sure, AI might be different from [Yet Another Cool AGPL Parser] - but then who has a bigger data moat and AI engineering talent than big tech, to just implement it for themselves?

** https://opensource.com/article/17/2/decline-gpl

Replies from: IvanVendrov, Daniel_Eth
comment by IvanVendrov · 2021-06-05T17:18:53.318Z · EA(p) · GW(p)

This is a helpful counterpoint. From big tech companies' perspective, I think that GPL (and especially aGPL) is close to the worst case scenario, since it destroys the ability to have proprietary software and can pose an existential risk to the company by empowering their competitors. Most of the specific clauses we discuss are not nearly so dangerous - they at most impose some small overhead on using or releasing the code. Corrigibility is the only clause that I can see being comparably dangerous: depending on the mechanism used to create future versions of the license, companies may feel they are giving too much control over their future to a third party.

Replies from: technicalities
comment by technicalities · 2021-06-14T04:22:17.505Z · EA(p) · GW(p)

I think I generalised too quickly in my comment; I saw "virality" and "any later version" and assumed the worst. But of course we can take into account AGPL backfiring when we design this licence!

One nice side effect of even a toothless AI Safety Licence: it puts a reminder about safety into the top of every repo. Sure, no one reads licences (and people often ignore health and safety rules when it gets in their way, even at their own risk). But maybe it makes things a bit more tangible like LICENSE.md gives law a foothold into the minds of devs.

comment by Daniel_Eth · 2021-06-14T01:49:08.276Z · EA(p) · GW(p)

I'm not sure how well the analogy holds. With GPL, for-profit companies would lose their profits. With the AI Safety analog, they'd be able to keep 100% of their profits, so long as they followed XYZ safety protocols (which would be pushing them towards goals they want anyway – none of the major tech companies wants to cause human extinction).

Replies from: technicalities
comment by technicalities · 2021-06-14T04:23:57.212Z · EA(p) · GW(p)

I think you're right, see my reply to Ivan.

comment by PeterMcCluskey · 2021-06-12T03:13:19.379Z · EA(p) · GW(p)

I see some important promise in this idea, but it looks really hard to convert the broad principles into something that's both useful, and clear enough that a lawyer could decide whether his employer was obeying it.