- Implementing AI systems will become more expensive as models become cheaper, experts warn
- Cost-efficient tokens are resulting in more complex AI agents
- Soaring costs could result in changes to how businesses rely on AI, as the inference paradox becomes more apparent
The drive towards agentic AI workflows could be about to hit its first real challenge: cost. With more companies using agentic AI over chatbots, the costs are set to mount up.
Analysts have raised the alarm on token economics, citing evidence that as the price of a token decreases, the resulting workflows are becoming increasingly complicated. Consequently, rather than driving agentic AI workflow costs down, increased token use is pushing costs up.
The research by Gartner warns of the realities of the “inference paradox,” which may lead to businesses reappraising their approach to AI development and use.
What is the inference paradox?
Gartner’s report, which has identified trends driving token economics, indicates that efficiencies in token price are being canceled out by the costs linked to increased AI capabilities.
They list a trio of trends: improving foundational model cost economics, improved AI efficiency enabling “more sophisticated AI applications,” and the higher use of AI tokens by “sophisticated workflows” compared with standard chatbots.
Gartner describes this as the “inference paradox,” the situation in which “better unit economics escalat[e] the overall cost of AI without providing a clear pathway to commensurate and predictable value.”
“The harsh economics of the Inference Paradox are exemplified by the differences between a simple chatbot and an AI agent,” notes Will Sommer, a Sr. Director Analyst at Gartner.
“Where a simple chatbot must read and interpret a query and quickly respond with a probabilistically reasonable answer, an AI agent must constantly reason, negotiate, and question itself.”
The study suggests that using an agentic reasoning model rather than a basic chatbot increases inference costs fivefold, a multiplier that can increase with complexity.
Cost management priorities
The implications of the inference paradox are not yet fully understood or realized by the companies driving forward with agentic solutions. As Sommer notes, “Product leaders cannot rely on more efficient token economics to rationalize AI costs.”
Various solutions may become apparent – such as corporations partnering over AI agent development to manage costs – but as the drive toward more complex models and agents continues, so the cost and volume of tokens will increase.
“Each successive generation of AI capability will necessitate more, and often more expensive, tokens,” added Sommer. “There is no reliable, economical one-size-fits-all model on the horizon. Producing competitive AI products will require developing and maintaining complex multimodel ecosystems.”
Given the potential destruction that going all-in on AI could wreak for some companies, it could be time reappraise that “cost cutting” agentic AI strategy.
