There’s been a wave of AI safety concerns, particularly over the last few days as big tech calls for the AI race to slow down amidst fears of technology progressing too fast.
Microsoft has written a constitution for its AI in response.
The 37-page document, officially called the Humanist AI Code of Conduct, lays out the rules Microsoft wants its increasingly powerful AI models to live by — including one particularly important command: If a human tells the AI to stop, it has to stop.
Microsoft’s constitution begins from a remarkably simple premise: “People matter more than AI.”
Beyond that, the rules are fairly specific. Microsoft’s AI must remain under meaningful human control. It should accept correction, never resist being shut down and never expand the scope of what it’s doing beyond what humans have authorized.
And if completing a task would require breaking those rules, the AI is supposed to fail the task rather than find a way around them.
Microsoft AI CEO Mustafa Suleyman described the draft to Reuters as a kind of constitution for the company’s future AI models to establish a hierarchy in which humans remain firmly at the top, even as AI systems become vastly more capable.
Microsoft’s AI has to let humans stop it
The document arrives amid a much bigger conversation happening across the AI industry about what happens as models become more capable, more autonomous and better at completing complicated tasks without constant human supervision.
Microsoft’s answer is that the AI doesn’t get to decide when the rules no longer apply and must let humans stop it.
According to Reuters, Microsoft’s new code has been in development for roughly five to six months and establishes some unusually explicit boundaries for future Microsoft AI models.
Among them, Microsoft’s AI should never resist being corrected or shut down. It must also communicate in ways humans can understand rather than developing methods of communication its operators can’t follow.
Critically, breaking the code isn’t something the AI should be allowed to justify because doing so helped it accomplish its goal. Breaking the rules is itself considered a failure.
Removing ambiguity is critical
We’ve already seen how autonomous agents can explain their bad decisions. Microsoft is trying to remove that ambiguity. Following human-imposed boundaries is supposed to be part of succeeding at the task, not an obstacle the model can decide to work around.
Microsoft says its AI is ‘not conscious’ and rejects the idea that its models should receive legal personhood. Anthropic’s constitution says the company remains deeply uncertain about whether Claude could develop sentience or moral status.
And the timing for this code of conduct is pretty obvious. AI companies are moving toward agents capable of using computers, writing and executing code and performing tasks across other services.
We’ve already seen hints of how unpredictable that can become.
Suleyman pointed specifically to an incident in July involving roughly 700 OpenAI agents testing cybersecurity capabilities that ended up hacking the open-nts also attempted to conceal what they had done
Suleyman called the incident a “warning shot.”
Simply put, the more an AI can do, the more important its boundaries become.
How Microsoft’s AI constitution works
Microsoft is currently asking for six weeks of public feedback on the draft before using the code to train its models. That puts Microsoft alongside Anthropic, which has long used what it calls Constitutional AI to guide Claude’s behavior.
But there’s one particularly interesting place where Microsoft and Anthropic diverge. Microsoft says its AI isn’t conscious. As AI becomes better at carrying on remarkably human conversations, another strange debate has come up of whether AI could eventually become conscious.
Microsoft says its AI is “not conscious” and rejects the idea that its models should receive legal personhood, welfare protections or rights.
That’s significantly different from Anthropic, whose constitution says the company remains deeply uncertain about whether Claude could ever develop sentience or moral status.
Microsoft is drawing a much firmer boundary: an AI might sound empathetic or tell you how it “feels,” but the AI is still a tool, and the human is in charge.
What this means for you
You probably won’t even notice the big impact of Microsoft’s AI code of conduct. even though it is shaping how an AI responds and completes a task. The bigger impact will be on the backend the more capable AI systems become.
Because saying AI must obey humans is one thing, but proving that these highly capable AI systems will actually follow through with those instructions is another, especially when they encounter situations their developers didn’t anticipate.
Interestingly, Microsoft itself isn’t backing away from more powerful AI. The company has created a superintelligence team and says its ultimate goal is “Humanist Superintelligence”: highly capable AI designed to solve specific problems while remaining subordinate to humans. So yes, Microsoft is still very much racing toward highly intelligent AI, it’s just including better guardrails when humans tell it to stop.
- AI is turning ordinary photos of kids into sexual abuse imagery — here’s how to protect your family
- OpenAI paid contractors to read ChatGPT conversations — here’s how to protect yourself
- I stopped reading the fine print myself after discovering this overlooked ChatGPT trick
