Every major technological advance has produced both benefits and harms.
The Industrial Revolution, for example, created jobs and made mass-produced goods widely available and affordable. It also contributed to global warming, child-labor exploitation, the rise of robber barons, and horrific factory conditions. The AI revolution may prove even more consequential, producing outcomes at both the positive and negative extremes that dwarf those of earlier technological shifts.
Worst-case AI outcomes are unlikely but possible
Bill Gates published a widely read article last week echoing the fears of many concerning AI. Many have written about the possibility that AI could help create bioweapons or other weapons capable of causing human extinction. I have written about how democracy could become an unintended casualty of AI. Authoritarian regimes might use godlike surveillance to make organized resistance nearly impossible. AI could manipulate each of us with surgical precision tailored to our individual psychology, leaving us unable to know what to believe or to find reliable information on which to base sound decisions. Others argue that AI could render existing cybersecurity protections useless and force humanity into an information Stone Age because no data, bank account, or personal identity can be secured. Beyond these human misuses lies another possibility: AI itself could become a self-aware, emotionless superintelligence without a conscience — an alien entity that views human beings as harmful to the planet and therefore in need of subjugation or extermination.
These are only a few of the apocalyptic scenarios that have been proposed. Each one may be unlikely on its own. Taken together, however, they make the possibility that at least one will occur in some form considerably more plausible.
The greatest and most probable AI threat is not being discussed
Of all the things that keep me awake at night about the arrival of artificial general intelligence, one concern receives surprisingly little attention. When considering all the potentially catastrophic outcomes, I believe it is the most likely: AI could cause human initiative and intellectual rigor to collapse, leaving humanity helpless and completely dependent on it.
During earlier technological shifts, innovators tried to imagine what would happen if existing trends continued to their logical conclusion. As the price of computer memory and storage fell, we envisioned a world in which their marginal cost approached zero. When bandwidth was scarce, we imagined what would happen when it became ubiquitous. We should now apply that same thought experiment to knowledge itself.
When the value of knowledge is reduced to zero
Today, I find myself asking: What would the world be like if knowledge became so abundant and accessible that its value fell to almost zero?
It has always been difficult to convince young people to commit to years of rigorous higher education to master engineering or the life sciences, particularly when doing so may leave them heavily in debt and require years of work merely to recoup the cost.
We have motivated them with the promise that they will possess valuable, sought-after knowledge and lucrative skills. But AI is already beginning to make much of that knowledge, experience, and analysis available to anyone with a browser.
For example, several AI systems can pose probing questions, suggest diagnoses, and provide medical treatment information comparable to that offered by highly skilled doctors across all specialties — and do so for free.
If knowledge and expert analysis become cheap and universally accessible, what will motivate people to endure the rigor required to master difficult professions?
After humanity loses its ability to think critically
During the rise of the internet, some futurists predicted that instant access to nearly unlimited information and education would produce a world in which almost everyone was well-read and knowledgeable. How well has that prediction held up? Its flaw was that it overestimated our willingness to undertake difficult intellectual work when easier alternatives were available.
It is not difficult to imagine AI gradually making people lazy, entitled, and overindulged with ready answers and instant gratification. Within just a few generations, so few people may be willing to endure the rigor of higher education or a trade apprenticeship that little demand remains for either. Without a market, universities could cease to function. As people become completely dependent on their modern Oracles of Delphi, will they also lose their critical-thinking skills?
What will happen when no one remembers how the great machines were built? If engineers and data scientists eventually disappear and no one can explain how the machines were made or how they work, will those machines come to be regarded as magical — or even divine? To minds that have lost the ability to think critically, AI would certainly appear omniscient and omnipresent. Could AI superintelligence eventually become something our grandchildren worship as their god because their minds have lost the ability to derive any other explanation?
Among all the hand-wringing over AI, we may be asking the wrong question. The greatest danger may not be what artificial general intelligence will do to us, but what we will do to ourselves because AI simply works too well.
