AI can do more and more.
At first, we mainly asked a chatbot questions. Now AI can write texts, analyse information, program and increasingly carry out tasks independently.
Such an AI system is often called an agent.
That is useful. But it also raises an important question:
If AI can do something, does that mean it should be allowed to do it independently?
A simple typing course
This question occurred to me while building TiepMiep, a typing course for my children.
Suppose TiepMiep discovers in the future that a learner regularly mixes up N and M.
The system decides to offer a few extra exercises.
Probably no problem. The consequences are small, and if it does not work, we stop.
But suppose the system discovers that many children struggle with N and M.
What may it do then?
Simply report something noteworthy? Propose an improvement? Adapt the course itself? Roll the change out to all children immediately?
Somewhere along that sequence, our reaction will probably change from:
“It is useful that AI does that itself.”
to:
“Wait a moment. Should AI be allowed to decide this itself?”
A digital colleague with different keys
Perhaps a simple comparison helps.
Imagine a new colleague joining a company.
On their first day, we would not automatically give them the office key, access to all customer data, permission to make payments and authority to change company rules.
Not because we distrust that colleague.
But because different responsibilities call for different permissions.
Why would we do things differently with AI?
AI that may view information does not automatically need permission to change it.
AI that can propose an improvement does not need permission to implement it itself.
And AI that may change something does not automatically need permission to decide that the change applies to everyone.
Being able to do something is different from being allowed to.
Not every decision has the same consequences
I think this contains an important starting point.
We often ask how much autonomy we want to give AI. Perhaps a better question is:
What do we want to give AI autonomy over?
An unsuitable extra typing exercise has few consequences.
Changing the course for every learner has greater consequences.
And independently modifying data or software calls for different safeguards again.
The technology may be the same. The consequences are not.
This does not mean a human needs to approve every AI action. Much of automation’s value would then disappear.
Within clear boundaries, AI can act independently perfectly well. As potential consequences grow, we can add controls or deliberately keep a decision with a human.
From capability to permission
In recent years, we have mainly been fascinated by:
What can AI do?
As AI acts on our behalf more often, a second question becomes at least as important:
What do we want to let AI do independently?
This is more than a technical question. It concerns trust, responsibility and control.
For now, my typing course will not be rewriting itself.
But this small example reveals a bigger principle:
The more keys we give AI, the more important it becomes to consciously decide which doors they may open.