Enterprises are still figuring out what it takes to govern the fast-moving AI being integrated into their tech stacks. Keeping pace with the technology while implementing effective governance is a career-defining challenge for CIOs, especially if enterprises must also cleanup work produced by AI.
“Is that code actually making its way into the products or the applications that we’re running? If it is, is it of good quality — or are we finding ourselves having to fix bugs that we’re introducing?” asked David Vidoni, CIO of Pegasystems. an AI decisioning and workflow automation platform,
The speed of AI’s deployment is creating a gulf in governance that teams are hard-pressed to keep up with. Machine learning and AI platform Dataiku surveyed 685 enterprise CIOs; 84% report employees are creating AI agents and applications faster than theIT department’s ability to govern them.
Further, less than half (47%) of responding organizations have clear governance controls in place, according to asurvey of 1,200 business decision makersfrom AI governance platform OneTrust.
As CIOs grapple with governance gaps, there is a growing expectation for demonstrable results from new AI initiatives. Just over three-quarters of CIOs (76%) surveyed by Dataiku think their role will be at risk if the company doesn’t show measurable gains with AI by the end of next year.
The clock is ticking, and governance gaps threaten to derail AI business value.
Enterprises still figuring out financial governance
The financial piece of AI governance is a considerable challenge for CIOs. Just 21% of CIOs have full visibility into AI agent and AI workload costs across different use cases and business units, according to the Dataiku report.
“My concerns are around financial controls and oversight: having visibility, knowing who’s spending, what they’re spending it on,” Vidoni said. With that concern, he also expressed a desire to see limits in place to avoid runaway activities.
Butfiguring out the costis just a part of financial governance. The enterprise must also look to cut excess spending and consolidate areas of redundancy or repetition — which can easily arise without sufficient oversight. In particular, spending can spiral out of control without a centralized repository for the AI tools and skills employees are developing.
“People are recreating the same skills over and over,” Mike Anderson, chief digital and information officer at cybersecurity company Netskope, said.
Lack of financial governance can amplify costs and work, eating away at the potential value CIOs are focused on achieving with AI.
Data quality governance is a work in progress
AI amplifies the data governance challenges that existed before LLMs, small language models and agents entered the equation, and the quality of the available data plays a critical role in the effectiveness of these tools.
“Sometimes, if you’re not curating that data properly, it can be referencing old information or incorrect information. And the more data you give it, the higher likelihood it could even hallucinate,” Vidoni said.
He and his team are kicking off an effort to inventory “all of the known and approved and recommended sources for data,” Vidoni shared.
But even proactive Inventory initiatives like that are complicated by the continuous influx of data. Complicating the matter is the AI slop generated in the form of not just documents but also applications
“Your data hygiene is not just what exists, but what people create,” Anderson said.
Effective AI implementation and scaling hinge on governance that ensures the quality of the underlying data. More than half of business leaders (52%) cite data quality, access, privacy or security concerns as the top factors delaying the launch and scaling of AI, according to the OneTrust survey.
Shadow AI complicates governance
Roberta Schwartz, EVP and chief innovation officer of Houston Methodist Hospital, deals with a governance challenge familiar to all enterprise IT leaders: the sheer volume of products in the tech stack. This can include AI developed internally and from third-party vendors.
“Everything that we’re purchasing or doing has upgrades in them that have some level of algorithm,” she told InformationWeek. “It’s hard to get any device that doesn’t have something called AI built into it.”
Governing that technology requires knowledge of where AI is being used, what data it is accessing, how it is using that data and if it is storing any data.
The third-party risk management frameworks and policies that enterprises already follow can be adapted to address AI. “Anytime we find a vendor that we feel is a cyber risk … we either lock it down or bring it back to them and say, ‘Here’s what we need you to do,'” Schwartz said.
But then there is the issue of shadow AI. Employees eager to experiment with and leverage AI are spinning up agents and using tools, either in violation of governance policies or in absence of any existing controls.
“We can’t have it be the Wild West where someone gets a bright idea, they spin up a MCP server that can make this particular application available, and then anyone can hit it with an agent,” Vidoni said. That could lead to performance issues, instability, and data leakage with incomplete controls on access.
Finding the right tools and people is not easy
Enterprises are in the midst of moving to a disciplined approach for scaled AI adoption. Getting there requires layering on the right operational governance.
“In an ideal world, I put a prompt in and there’s something that intercepts that prompt and says, ‘Let me route you to the right tool to answer that question and puts people in the right place,'” Anderson said. “That doesn’t exist today. Until it does, it’s all about education.”
People in an enterprise need to learn how to choose the appropriate model, and that kind of education takes time. It’s also an ongoing journey; more models and tools continue to come out and the pressure is on to use them efficiently to increase revenue. This requires the human in the loop to ensure AI models, tools and agents deliver accurate, unbiased outcomes.
As agents become more autonomous, governance needs to be in place to recognize and prevent authority drift and behavior deviation The standard line from enterprises is to always have a person be ultimately accountable for AI outcomes. But figuring out accountability in reality is a relatively nascent element of governance
Two-thirds of CIOs lack confidence that they could show an end-to-end audit trail for an AI agent’s decision if asked for one by regulators, according to the Dataiku survey. And those humans in the loop can only move so quickly.
“One of our big concerns is human fatigue because we have machines working at machine speeds,” Anderson said. “All of a sudden, we start seeing people just rubber stamping because they’re just trying to keep up.”
Getting on top of governance challenges
Prioritization has become even more important to governance as the demands of AI are layered on top of legacy infrastructure. Managing the expanding set of governance challenges and closing the gaps requires collaboration at the leadership level and throughout the enterprise.
Governance committees are frequently the nexus for this collaboration: Identifying where governance is needed and developing the policies and procedures to support it. It is between the writing of these policies and procedures and their application that small cracks can widen into governance gaps.
“When we think about written policies and education, it’s only as good as the people who consume it,” Anderson said. “I think that’s the biggest weakness we have in governance right now.” Despite its complexities, governance must be applied in a way that makes it understandable for the people responsible for following the policies and procedures. “The more complicated you make it, the more difficult it will be to follow,” Schwartz said.
For example, employees need a straightforward way to report instances when AI provides an incorrect answer or behaves unexpectedly. “Who do you report to?” said Schwartz. “You need an easy pathway rather than me having to log into a separate computer.”
In the long-term, more tools to support the financial, data quality, privacy, security and operational aspects of AI governance will become available. But the governance challenges won’t disappear.
“My biggest concern long term … is over-reliance on AI and the <a href="https://myappsplus.com/nyt-strands-hints-and-answers-for-friday-october-9-game-950/” title=”NYT Strands hints and answers for Friday, October 9 (game #950)”>answers it’s providing,” Anderson said. “We have to remain curious, and [not] forget all the things we’ve learned over the years.”
