AI promises to cut code so fast that applications can be brought to market quickly. In May, 82% of application developers reported spending less time writing code with the help of AI, according to a study published on arXiv.
But there’s a problem: The QA process hasn’t kept pace. While AI enables developers to build features in parallel, QA checkouts remain largely serial. The functions of new AI applications are constantly altering underlying test data, which skews results and invites inaccuracies.
Build This Now, a provider of predefined hooks, agents, skills and workflows for AI development, shared data with me from its own benchmark testing.
“We could not reliably run more than about four features in parallel before it turned into a mess,” they said. “The build agents kept up fine. The verification [i.e., QA] did not.”
Build This Nowwent on to explain that past roughly four concurrent features, three things went wrong:
-
Test runs started colliding.One feature’s test setup could disrupt another’s teardown when they used the same database rows, ports or other shared resources.
-
State drifted.The environment the tests assumed no longer matched the environment that existed, because a sibling agent had changed it underneath them.
-
The testing agents spun into loops.They re-ran and rechecked the same flows without converging, burning time and tokens, unsure whether a failure was real or a collision.
Still, there are a number of steps that developers can take to improve the speed and accuracy of QA to better keep pace with AI coding.
What developer and QA teams can do
-
Improve AI modeling.When enterprises construct strong AI models in the first place, it’s easier to QA them because they align with the business use cases they are developed to address. That’s because strong AI models use high-quality data, and the team developing them thoroughly understands how <a href="https://myappsplus.com/best-buy-has-lgs-most-affordable-2026-oled-tv-at-300-off-right-now-more/” title=”Best Buy has LG's most affordable 2026 OLED TV at $300 off right now, more”>best to plumb this data — whether through algorithmic decision trees or queries, neural network training and execution, or some other methodology prescribed for the use case. Keep in mind that model development can be challenging for new and inexperienced AI teams; outside consulting or external help can be beneficial the first time through.
-
Limit the features you test in parallel to four.TheBuild This Now study found that testing more than four AI features at once dramatically reduces QA effectiveness, so limit parallel AI feature tests to no more than four whenever possible. If an AI project requires more than four features, which it likely will, try to arrange QA testing as each new set of four features is added. There could be some issues with QA accuracy because the underlying data that originally existed for testing has already been modified by previous feature tests, but at least you have a phased and controllable testbed.
-
Accept the reality that data, app tuning and QA will be ongoing with AI.Even if your data is picture-perfect and of the highest quality, the original business use case and the data may change over time. When this happens, the AI accuracy can diminish. It’s easy to see when this happens because users will rely less on data, and there will be noticeable declines in the accuracy of AI outcomes. The takeaway is that AI model, data and application tuneups will be needed, and QA will be an ongoing activity. This represents a change in work practice for IT end-user project teams, but it is also a change in perception for upper management, which has historically expected projects to finish. CIOs and other AI leaders must impress upon management that AI systems are ever-evolving.
-
Knock out every AI QA bottleneck you can find.The single-threaded nature of QA already makes it an AI bottleneck, but some issues can be avoided. For example, IT QA teams are typically understaffed, but you can’t afford this with AI. If you have only one QA person for every six AI developers, there will be a resource bottleneck. And if you see the AI development cycle taking three days and the QA cycle taking five days, you also know you have a resource problem. The solution is to beef up the QA staff so it can handle the load.
-
Use automated test tools.Over the past decade, QA staffs have adopted automation to autogenerate and run test scripts. This eliminated many of the routine testing checkouts that staff had previously had to perform manually. A similar approach can be adopted for AI system checkouts. Instead of designing test cases for the AI, the focus would be on defining criteria for various test scenarios and determining which process variants must never fail. AI automation tools for QA can also automate tests and traces of entire AI-user flows within a business process to ensure everything is working at every point. Finally, AI test automation can generate test scenarios and verify data quality. All of these capabilities serve to offload tedium so QA staff can focus on larger issues.
How is your organization balancing the speed of AI with the QA process? Let us know at[email protected].
