New report finds AI may not be the problem, but the missing context needed for it to work
Professionals rarely blame artificial intelligence (AI) tools when they fail to deliver at work, and are instead pinning the failures on missing or inaccurate information the technology needed, according to a new report.
Data released by Latch revealed that only eight per cent of respondents said the technology was not capable enough when an AI project fell short.
On the other hand, 27% of respondents said the main reason the AI project fell short was inaccurate, incomplete, or unavailable information.
Another 17% said this was because the system didn’t understand how the work is done, while 13% said this was because AI couldn’t connect with existing business systems.
“Together, those three reasons account for 57% of respondents whose project fell short. All three point to the same gap: AI didn’t have the context it needed to do the work,” the report read.
Missing context is the problem
This problem of missing context comes as the report underscored how it can be difficult to put on paper what a job really entails.
Nearly three in 10 of executives and employees (29%) said only between 26 and 50% of their recurring work is documented well enough to hand off.
And even when written instructions exist, only 22% of the respondents said their written instructions completely match how the work is actually done.
AI’s blind spot is also unwritten rules and exceptions, according to the findings. Nearly a quarter (24%) of the respondents said one barrier to automating more work is the share of work that depends on exceptions or human judgements.
The result is the need for a “babysitter” for AI tools that are expected to carry out the work, according to the findings.
While uncaptured knowledge does not stop AI from working, the report noted that it “changes” what working means.
The impact: 73% of respondents whose organisations use AI automation said work handled by AI or automation needs to be corrected, redone, or finished by a person at least sometimes, while 29% said this happens often or very often.
What should be done?
The answer to this problem is less about buying smarter AI and more about getting better at “capturing” the work, according to the report.
It urged employers to treat AI failures as context failures first to avoid immediately writing off an AI automation.
“Respondents rarely blame the technology. They point to missing information, systems that didn’t understand the work, and AI that couldn’t connect to existing systems,” the report read.
“Before writing off an AI automation, uncover if it had the context for success in the first place.”
The report also underscored the importance of bringing the people “closest to the work” into AI decisions.
This means capturing the scope of the work from the people who actually do the work, especially before they leave or retire.
The report notes that half of employees aren’t involved in AI automation, and about one in four didn’t name a task they’d hand to it.
“They are also the people who know the unwritten rules and exceptions that AI will have to learn. Involve them in choosing what to automate and in checking what AI produces,” the report read.
