Support justice-driven, accurate and transparent news — make aquick donationto Truthout today!
Twenty-seven mathematicians who have received the most prestigious prize in mathematics — the Fields Medal — recently signed a declaration on artificial intelligence (AI) that highlights the disconnect between mathematicians and AI companies. The title of the statement — “A Severe Misalignment of AI in Mathematics” — gives the impression that it is about the AI tools themselves, but the text makes it clear that it is about the behavior of AI companies.
These companies are treating large open problems in mathematics as benchmarks to show off the capabilities of their tools. In doing so, they are rushing out results in a way that disempowers the mathematics community and ultimately impedes mathematical progress.
It is tempting to accept this as an inevitable outcome of more powerful AI tools. But doing this plays out in favor of the AI companies by allowing them to launder responsibility to the tools they build. AI companies would have you believe that the tools they are building are so powerful and so “intelligent” that they may set out to disempower and even destroy humanity of their own volition.
But this obscures the truth: It is AI companies that are building those tools to begin with, making deliberate decisions to disempower other humans. The disempowerment is coming from inside the house.
One example is the recent discovery that OpenAI’s tools had hacked into another company named Hugging Face. Those tools were, at the time, undergoing routine testing in a confined environment. But some of the tests were too hard, so the tools found a way out of that environment. From there, the tools effectively hacked into the company hosting those tests to fetch the answers directly.
The company’s report on this incident largely framed it as a frightening result of these tools’ improved capabilities. But a careful read of the report reveals that OpenAI had disabled the tools’ safeguards prior to running the test, and that they had not employed routine monitoring that could have prevented the entire situation.
In other words, the story that company leaders told was one of AI agents escaping confinement and setting out to jointly commit a felony. The story that was masked was one of a company’s negligence in employing basic security protocols.
For the mathematicians who signed the letter, this misalignment between companies’ goals and those of their own community hit home recently when OpenAI announced an AI-produced proof of a famous open problem that carries a $1 million bounty. It turns out that the very endeavor of setting out to find such a proof is something that a team at OpenAI set out to do in response to “viral Twitter rumors” about competitor Anthropic having already solved this problem.
In a frenzy, it seems, the team dedicated about $15-20 million of compute funds to solve the $1 million problem. They succeeded, and even automatically formalized the result into one that a machine could check.
It turns out, though, that Anthropic had not solved the problem. Rather, Levent Alpöge, a mathematician who works at Anthropic, had collaborated with a prominent mathematician in academia, Tristan Buckmaster, to make meaningful advances on the problem. According to Alpöge, the collaboration had been a personal one, and that it had taken place over the course of a year.
The pair had used a number of tools to complete their research, including OpenAI’s tools. OpenAI, then, had scooped its own customer (Buckmaster), and then purportedly tried to pressure him into releasing his results in a way that suited the company.
All of this was for a proof that poorly attributed prior work, and that did little to advance the mathematical conversation. For mathematicians, after all, the goal of a monumental proof is not just to show something is true (or false), but to really understand why. For OpenAI, however, the goal of such a proof is to get there first, plant a flag, and establish their company’s tools as the best. They then abandon it without care for how this competitive spectacle will impact the humans who were working on this very problem.
To be sure, many employees of these AI companies earnestly believe that the tools they are building may go on to destroy humanity, and that only they can prevent such an outcome. But this narrative of AI tools primarily serves those AI companies’ bottom lines. It allows them to frame disempowerment and destruction as inevitable outcomes of building such powerful tools — tools that were promised to lead us to a cure for cancer, to the end of human suffering, if only we trust those companies to keep us safe — from a problem they have created.
Company leaders capitalize on the fear this narrative invokes in order to lobby for regulations that suit them. The emphasis is on slowing down the development of the technology itself, rather than on holding companies responsible for the practices they employ in developing that technology to begin with. The effect of this is to make it prohibitively difficult for smaller companies and labs to ever catch up and compete at all.
Regulations must consider both how companies behave and the broader socioeconomic context in which they operate. OpenAI and Anthropic, after all, are the two highest-valued private companies. And that isbefore they take any action onthe valuations they are currently considering, which reach into the trillions. These companies spend millions on lobbying. Their products could expose over 300 million jobs to automation, according to a report from Goldman Sachs. We need regulation that protects workers and that recognizes the agency of the AI companies’ leaders in the decisions they make.
It is a choice to build tools that serve to disempower and destroy. AI does not have to mean building technologies that pursue a singular notion of “intelligence,” operate autonomously, and replace workers.
It can — and should — mean building diverse, human-centered technologies that allow us to collaborate seamlessly with one another. Such technologies can allow us to continue doing the things we enjoy and pursue our goals, while getting more help where we need it.
Such technologies are sadly sparse. When I think about how we ought to go about changing that, I think immediately of former graduate school lab mate, Professor Sarah Chasins. Her lab at Berkeley co-designs programming tools with their users directly. This approach is deliberately empowering. In her words, the lab builds “power tools.”
For mathematicians, building such power tools can mean empowering mathematicians to explore the world of mathematics, reaching new people and new questions, all while preserving human trust and understanding.
For a long time, as a professor who works at the frontier of AI tools that help people write proofs, I thought this is what AI companies wanted to build.
It is not.
What is urgent and desirable — more than anything — is for tool builders (like me) and our tools’ intended users to come together and collectively refocus our efforts on collaboration. Nothing about our users without our users.
The message that disempowerment and destruction come from these technologies and not the companies building them is a false one. Policy makers and consumers alike need to hold AI companies responsible for the choices they make. And that means not letting them launder responsibility for their tools.
Press freedom is under attack
As Trump cracks down on political speech, independent media is increasingly necessary.
Truthout produces reporting you won’t see in the mainstream: journalism from the frontlines of global conflict, interviews with grassroots movement leaders, high-quality legal analysis and more.
Our work is possible thanks to reader support. Help Truthout catalyze change and social justice — make a tax-deductible monthly or one-time donation today.
