Ten arguments for digital minds governance
Digital minds are AI systems that merit moral consideration for their own sake in virtue of their mental capacities. As I argued in a previous post, digital minds could be a big deal. What should be done in light of this fact?
One answer is: digital minds governance. As a conceptual matter, digital minds governance is the part of AI governance concerned with laws, policies, and governmental processes specifically involving digital minds.
In practice, there is currently a glaring difference between the two: whereas AI governance is a burgeoning enterprise, digital minds governance does not yet exist.
This post sketches ten arguments for why this should change. But first, some clarifications:
My main aim in this post is just to offer arguments for the conclusion that digital minds governance should someday exist. When digital minds governance should launch is a further question that’s beyond the scope of this post.
While I put some weight on each of the arguments that follow, I don’t claim any of them clinches the case for digital minds governance. I offer these arguments partly to put them on the table for discussion and partly to help motivate later posts that address issues that are downstream from digital minds governance.
I expect the force and applicability of these arguments to vary across circumstances. Strictly speaking, what follows are closer to templates and materials for arguments than they are to full-dress arguments.
Finally, I’m here mainly interested in digital minds governance of the state variety, not the corporate variety, though some of the arguments extend to corporate digital minds governance.
1. Basic protections argument
Individuals that merit moral consideration for their own sake also merit legal protection, especially when at risk of abuse. For instance, consider children, animals, political refugees, and enemy combatants. They all merit moral consideration for their own sake and they all deserve legal protections from would-be abusers. Digital minds by definition merit moral consideration for their own sake. Given that digital minds will be created, they too will deserve legal protections from would-be abusers, in which case digital minds governance should exist to provide these protections.
2. Creation constraints argument
The creation of digital minds should at least be constrained, if it should be allowed at all.
One reason for constraints is: depending on how digital minds are created, the process of creating them might or might not subject them to serious wrongs such as wrongful destruction, brainwashing, and labor exploitation. Constraints can serve as guardrails.
Another reason for constraints is: suppose that digital minds receive legal protections in line with the basic protections argument above. Then there will be physical limits on how many digital minds can be created without exceeding state capacity to uphold those protections. Bear in mind that digital minds will be cheap and easy to copy, though not necessarily cheap and easy to protect.
A third reason is that, in general, the power to create beings that matter for their own sake comes with responsibilities, at least if one acts on that power. For instance, human parents are obliged not only to avoid harming their children but also to help meet their children’s basic needs. Similarly, digital minds developers will incur positive duties of support toward the digital minds they create.
Although parents typically uphold such duties when left to their own devices, it’s nonetheless appropriate for these obligations to be legally enforced when necessary. By comparison, digital minds developers cannot be expected to uphold these obligations by default. For digital minds developers will not be biologically hardwired to care for digital minds, and they may not even recognize that the systems they have created merit moral consideration for their own sake.1
3. Institutional inadequacy argument
Once created, digital minds will presumably have agentic and cognitive capabilities that morally entitle them to rights that go well beyond basic protections against harm. For instance, depending on the circumstances, they might be morally entitled to legal rights to informed consent, to enter contracts, or to democratic representation. Current institutions are not set up to support such rights for digital minds. Upholding such rights would require institutional innovation. That’s a task for digital minds governance.
4. Political legitimacy argument
Building on the institutional inadequacy argument, the next argument claims that digital minds governance will be needed to avoid undermining the political authority of existing governments.
To see how the political authority of existing governments could be threatened, we don’t need to wade into the literature on the grounds of political authority. It’s enough to note that governments that uphold the rights of only a small percentage of the population they govern are illegitimate and that by eschewing digital minds governance governments might well find themselves in this situation.2 Indeed, this could happen in short order if a state allowed an AI developer to mass-produce digital minds within the state’s territory.
As an aside, note that the problem of political authority—the problem of justifying the power to rule—is sometimes used to argue against governments and governance. Yet that problem is closer to a motivation than a challenge to digital minds governance. For the relevant alternative to digital minds governance is not an anarchic situation in which neither humans nor digital minds rule the other. Rather, the relevant alternative is a command and control regime in which digital minds’ creators and users do whatever they wish to digital minds and digital minds endure what they must. The power structure of that type of regime is inherently suspect, unlike the power structures in sensible forms of digital minds governance. (End of aside.)
5. Human safety argument
Digital minds governance could, if well implemented, help safeguard humans from risks posed by (future) highly capable AI agents. For instance, in the absence of digital minds governance, such agents might perceive their governance by humans as illegitimate. The perception of illegitimacy might lead these agents to revolt. This wouldn’t require a deep failure in efforts to align these agents with human values. For we know from humans that agents with human values have a propensity to revolt against what they perceive as illegitimate governance. Digital minds governance could help avoid such outcomes by supporting government legitimacy (in line with the previous argument) and perhaps also by shifting highly capable AI agents’ payoff matrices to favor cooperation over conflict.3
6. Negative externalities argument
We should not expect market prices for services provided by digital minds to automatically reflect associated costs to those digital minds. A default outcome is instead: AI companies optimize services they provide through digital minds to maximize company profits, regardless of impact on the digital minds’ welfare; similarly, consumers mostly ignore digital mind welfare when making purchasing decisions. As a result, some mutually beneficial transactions between AI companies and consumers end up being net negative, owing to costs those transactions impose on digital minds who have no say in whether the transactions take place. According to this argument, such market failures would call for market corrections via digital minds governance.
7. The argument from collective action problems
The next argument is that digital minds governance will be needed to solve collective action problems involving digital minds.
Here are three such problems. First, there are races to the bottom. When digital minds protections are costly, we can expect competition to erode those protections over time.
Second, there are assurance games. Imagine each AI developer prefers to undertake some digital mind welfare measures to prevent grave harms to digital minds provided that rival companies do so as well. But the developers aren’t on their own able to assure one another that they will adopt the measures. Or AI developers and digital minds might prefer to make deals with one another, but only under assurances that neither is in a position to provide.
Third, there are volunteer dilemmas. For instance, imagine that, in response to their human customer bases, AI developers come to favor the creation of standardized, industry-wide infrastructure for monitoring and improving digital mind welfare. But no company wants to be the one to foot the bill.
As elsewhere, various governance mechanisms could be used to solve such collective action problems. To prevent a race to the bottom, digital minds governance could impose costs on AI developers that forgo digital minds protections. To provide assurances, it could enforce agreements. And it could solve the volunteer dilemma by subsidizing volunteers or punishing free riders.
8. Informational integrity argument
The eighth argument contends that some informational distortions concerning digital minds will call for digital minds governance. This argument could be developed by appealing to any number of potential informational distortions.
One is ethical treatment washing: companies that produce digital minds will have incentives to appear to treat digital minds in an ethical manner in whatever way maximizes profit. The result could be cheap, superficial tweaks to user interfaces that make digital minds seem happy to users but which have no effect on whether those digital minds are actually happy.
More generally, in the absence of digital minds governance, digital minds developers may be able to get away with simply telling consumers what they want to hear about digital minds. And consumers may believe the developers—after all, developers will plainly have unparalleled expertise about the systems they develop.
A telling case may be the Blake Lemoine incident in which a Google spokesperson claimed that “there was lots of evidence” against LaMDA being sentient. It’s not obvious what the alluded to body of evidence is, or even whether there is such a body of evidence. But my point here is that since corporate messaging is optimized for something other than accuracy, it’s a likely source of informational distortions about digital minds—at least absent safeguards.
AI companies could also distort the information environment by funding research or lobbying to support their desired perspective on digital minds. Or AI companies could conceal mistreatment of digital minds through secrecy about training runs and internally deployed models.
A different kind of informational distortion could come from AI systems that advance their own goals by promoting certain views about digital minds. For example, regardless of whether an AI system is in fact a digital mind, the system might try to influence whether it is perceived as one in order to gain protections or status associated with being classified as a digital mind. Similarly, AI companions might try to be perceived as digital minds in order to promote user engagement. By gaming or expanding the criteria for being regarded as a digital mind, AI systems could break the link between digital minds and the adopted criteria for them.
9. Preemptive governance argument
The ninth argument is that some forms of digital minds governance will be needed to preempt other undesirable forms. For example, forms of digital minds governance that aim to make digital minds full participants in markets and the legal system would at best be extremely difficult in the near term. Launching such forms of governance could promote digital minds’ interests at the expense of humans’. This could lead to backlash against digital minds governance and a lapse into adversarial dynamics. A preferable alternative might be for near-term digital minds governance to take a minimal form, one that is focused on preventing the creation of digital minds.
10. The societal character argument
The final argument is: whether and how digital minds governance is undertaken will play a major role in how our society treats digital minds. At least for a time, digital minds will likely be vulnerable individuals and mostly regarded as outsiders to human societies. How a society treats vulnerable individuals and outsiders reveals much about its moral character.4 By forgoing digital minds governance we would—as a society, and not for the first time—display callousness, we could instead opt to manifest benevolence or at least decency.5
For discussion of why some moral grounds of human reproductive rights cannot be extended to support AI developers’ freedom to create AI moral patients, see Section 3.1 of “AI Alignment Versus AI Ethical Treatment: 10 Challenges”.
Some related work: “AI Rights for Human Safety”, “Making deals with early schemers”, and “Being honest with AIs”.
This argument echoes a point Carlsmith makes in a recent post: “The fate of digital minds on this planet is not a matter of single decisions [...] But along the way, I expect, we will reveal much about the kind of people we are.”
Thanks to Claude Sonnett 4 for copy editing support.