Standard Cybernetics’ Jewkes Says the 99.95% AI Automation Beats PhDs at Scale — and Judgment Is the Undervalued Asset

Add preferred source
Matthew Jewkes, founder of Standard Cybernetics, argues that growth-stage companies chasing AI should stop keeping humans in the loop and instead build “full automations” that route every error, exception, and undecided case off the happy path. In a Clarvate deployment handling tens of thousands of tasks a month, his team hit 99.95% accuracy (“three and a half nines”) on the business-critical task set — up from 80–83% in early agent-style prototypes and ~95% once ambiguous cases were stripped out. The catch: the climb from 95% to 99.95% is expensive hill-climbing, and he estimates the team spent at least as much time building case routing as tuning the final accuracy. His larger bet is that cheap execution makes organizational judgment the scarce, underpriced input — so the durable advantage won’t be picking the right orchestration software in 2026, but managing collective human attention. The piece covers his three-bucket failure taxonomy, his selection criteria for automation candidates, and the “exception exhaust” he says is the real prize.

Key Elements

If you want to know where the money is in enterprise AI right now, Matthew Jewkes would point you somewhere uncomfortable: the exceptions nobody wants to look at. His team built an automation for Clarvate, which maintains an annotated history of the drug-development industry, and it now processes tens of thousands of new indexing tasks a month at 99.95% accuracy on the work the business cares most about — what he calls “three and a half nines.” The system, he claims, outperforms PhD-level humans at scale.

But the number that should stick with you isn’t the 99.95%. It’s the 3–5%. That’s the share of tasks that were exceptions or escalations requiring human attention — and speaking on the AI Engineer podcast, that small bucket is where the real strategic value hides

Cheap execution makes judgment the scarce input

Jewkes opens with a cost argument that reframes the whole AI debate. Raw white-collar output is “dropping… cratering,” and everyone he spoke with at the conference agreed existing capacity will get very cheap regardless of what happens with frontier models. The problem is what that does to the other side of the equation.

“Cheap execution increases the load on judgment on deciding what to do,” he said. “Organizational judgment was already in short supply, and this is going to make it even worse. I think this is currently underpriced.”

His description of how domain experts actually spend their weeks is grimly familiar: attention “smeared over multitasking,” loaded with complicated situations, making decisions that “could have been routine or obvious in retrospect,” with only a small fraction left to move the business forward. These are the people sales goes to when a new feature would close an account, or who get pulled into urgent customer tickets.

This is why he’s contrarian about the agent hype. “When I see an expert or a leader responding to agent notifications, maybe on their phone or on their laptop, I’m often suspicious,” he said. “Is that actually the best use of their valuable time and attention?” In his experience, agents are hard to maintain when skills and harnesses change, fail in “unusually shaped ways that enterprises find hard to work with,” and — worst of all — increase the decision burden rather than relieving it.

Full automations beat agents because they are legible

Jewkes’s core distinction is between agents and what he calls full automations. A full automation takes an input, produces an output, hides the complexity, and is organizationally familiar — human organizations have built software automations for decades (marketing, DevOps) and industrial automations for centuries. He calls them “one of the key tools by which human organizations can do bigger and bigger things” and “an escape hatch” for the whole system.

The AI-specific twist is using agency to reduce variance rather than to act autonomously in the main flow. An AI layer pre-filters inputs: checking whether a task actually matches the automation’s happy path, rejecting adversarial situations, and escalating anything that looks business-critical rather than normal-course.

“This is the use of AI agency to reduce the variance of an automation to make things more smooth, which thereby lets us expand the scope of what we can automate,” he said. He frames the whole exercise as “engineering agency out of the happy path.”

His stated benefits are three: improved margins, reduced organizational complexity by encapsulating whole chunks of work, and recovered expert judgment that can be routed to more important questions. That third one, he argues, is the most undervalued.

The Clarvate case: 80% to 95% to 99.95%

Clarvate holds an annotated history of the drug-development industry — tens of millions of documents, hundreds of thousands of edits a year, and an ontology with tens of thousands of indexing dimensions. The automation Jewkes built indexed documents and the relationships between them, handling tens of thousands of new tasks a month triggered by edits, insertions, or other changes.

The build started in 2024 with enthusiasm for o1 and o3-era models. His team built something resembling a modern research-and-assignment agent and hit 80–83% after a few sprints — exciting for a team used to spending months fine-tuning a BERT model to do one component of one pipeline. They compiled a thousand evals and sat down with domain experts.

One practice he singles out as effective: arriving with a large set of worked examples rather than asking a busy expert to fill out 100 evals in a blank spreadsheet, which “doesn’t create momentum.” Bringing the failures got the experts on board and fixated on where the system broke.

Three buckets, and the routing that took as long as the accuracy

The eval review sorted failures into three buckets, which became the design foundation:

Here’s the counterintuitive part that most engineering teams will underestimate. “I think probably in retrospect we spent at least as much time building case routing as we actually did on the final error climbing for the sort of critical set,” Jewkes said. The plumbing that decides where a case goes ate as much effort as the model tuning that decides what the model says.

The last four decimal places are where the money goes

Once ambiguous and unusual cases were removed, standard approaches reached about 95% on the critical set. The last stretch to 99.95% was “a whole bag of tricks” — hill climbing, reading large volumes of evals, and standard techniques for squeezing performance out of LLMs. He believes 99.95% is achievable for most automations when inputs are well structured and the case set is articulated from the business.

“If your inputs are well structured and you’re looking at a specific set of cases that you’ve articulated from your business, getting to three and a half nines is possible for most automations,” he said.

Two items from his tricks list he flags explicitly: read and write evals, and build a thousand evals while expecting to spend real time reading them. The reading is exhausting — parsing sycophantic model text to work out exactly why the AI did not do what was expected.

The cost warning is blunt: AI engineers are expensive, having one spend hours reading an eval spreadsheet is expensive, and the 95-to-99.95 climb is expensive. Beyond engineering, refactoring a non-engineering process brings organizational change, trust building, and ownership change. He argues an investment of that size should be paired with sales and marketing work to convert it into revenue, customer satisfaction, or expansion into a neighboring market.

The human comparison matters here too: he claims the system beat PhD humans at scale, but not a PhD on a single case with full detail and no fatigue or time pressure. That’s the honest boundary of the benchmark.

Pick scaled, specified, detail-oriented work — not startups

Jewkes’s selection criteria are specific. Build full automations where the domain is scaled and already has material ROI — a bucket of tasks tied to seven figures of revenue, ideally with multiple customer tickets signaling known pain, which “a CFO can understand.” Do not build them at a startup, in discovery, or in new businesses, where human flexibility is worth keeping.

Ideal processes are already running, have written specifications, and are detail-oriented. He doesn’t worry much about whether the specifications are good — specifications are hard, and the existence of momentum to write one shows the team believes the work is specifiable rather than raw judgment. Detail-oriented work is ideal because it takes expensive, memory-heavy work off humans, who fail at it at scale.

Subjective judgment work is the line. “AI to automate that specific differentiated process is still in my opinion… not where the action is,” he said.

Exception exhaust is the real prize

The third benefit Jewkes emphasizes — and the one he thinks gets overlooked — is that routing organizes the automation’s “exhaust” so experts can pay attention to it in a structured way. After routine work is cleared, an expert receives a pile of undecided and ambiguous cases, a pile of exceptions, and a small number of critical cases the automation was unsure about.

The ambiguous pile has two uses. First, it can pressure the organization to specify, since many ambiguous cases are only marginally valuable. Second, an expert facing a large pile can advocate to permanently freeze that work — deprioritize it until a business case appears — which is itself a reduction in organizational complexity.

The exception pile is where he sees the most value. Groundbreaking work like the Verve 102 heart-disease drug is only about 3% of new drug-development work; most of the field pattern-matches. An expert seeing a given unusual case once a week or every other week scattered across their queue cannot build intuition for its importance, cost, or who would pay for it. Bucketed in aggregate over two or three hours, patterns become visible and the expert can propose a specific investment — and when a real customer arrives with that problem, the expert already has a formed opinion. He says this happened organically on an eight-figure account.

He generalizes: most scaled business processes generate leading indicators of market demand that look like exceptions, and infrequent random encounters are not enough signal to act on.

He reaches for Rockefeller to make the point: “If your job is specifiable, get someone else to do it. What we want you to be doing is thinking, applying your judgment, looking for new opportunity.” He paraphrases it more bluntly elsewhere: “As soon as you can hire someone to do your job. Sit down, kick up your feet, think about how to make the company more money.”

The hope is that AI routes attention toward the most valuable places, so that thinking happens first.

Orchestration software won’t be the moat

Jewkes’s closing prediction is the sharpest for investors: the market will sort out the orchestration layer and the right harnesses, and there will not be a large differentiated advantage in picking the right AI orchestration software in 2026. The companies that succeed most with AI will be those that best manage collective attention and judgment, because the premium on that only goes up — for as long as human judgment holds a premium.

That pie is the whole thesis in one image: nearly all the volume is happy path, and nearly all the strategic upside is in the sliver.

The unresolved tension is cost against payoff. The Clarvate build was a major capex project justified by seven-figure revenue exposure, and Jewkes concedes the last four decimal places of accuracy are expensive to buy. Whether the attention-routing benefit — harder to measure than accuracy — justifies that spend at a typical growth-stage company is left to the listener, as is the question of how long the judgment premium he bets on will actually hold.

For anyone allocating capital or engineering time in enterprise AI, the practical read is this: the spread between the 95% demo and the 99.95% production system is where most companies stall, and it is a routing and judgment problem more than a model problem. The winners over the next two years may not be the firms with the best orchestration stack — they may be the ones that figured out how to capture the 3–5% of cases that don’t fit, and turn that exhaust into the next investment thesis.

Full content available at:Engineering Agency out of the Happy Path — Matthew Jewkes, Standard Cybernetics

• AI Is a Tailwind for Japan’s Economy, BOJ Governor Says, Also Flags Supply-Side and Cyber Risks
• China’s Shenzhen-based AI social startup Nature Select raises over $100 million in Series A+ funding, with 5Y Capital, Qiming and others co-investing
• Meta Buys AI Coding Startup Conductor as Agent Tool Race Heats Up
• Gates: Four Professions Withstand AI—But Millions of Jobs Are at Risk
• Washington Think Tank Warns of ‘Sovereign Agents’ That Could Outrun Human Control