AI in software engineering has moved from being an autocomplete tool to “Agentic AI”, where these AI systems are actively reading our environment, coming up with multi-step plans, hitting APIs, and modifying cloud infrastructure on their own.
This speed of AI comes with risks at enterprise scale. An agent that went rogue can trigger a domino effect of system failures or blow through our AWS budget while you are away from your desk, getting coffee.
The Safe Speed Limit for AI Autonomy
I would like to use the Self-driving cars framework to express the level of AI autonomy in the systems.For enterprise systems, right safe speed limit sits at Level 3 (Conditional Autonomy). At this stage, AI can handle multi-file changes and run loops within strict boundaries, but a human still has to sign off and pushing past this will end up in troubled waters.
Recent reports/studies show AI agents hallucinate software packages about 20% of the time. Because these models are essentially just guessing the most probable next word, they need to be boxed in by hard, deterministic rules.
1. Infrastructure (GitOps): Guarding the Deployment Layer
In GitOps, our Git repo is ourour live cluster drifts away from that baseline, but letting AI to fix things blindly, will end badly
-
What you should automate: Use AI as an investigator. When drift happens, the agent should spot it, figure out how bad it is, and draft a pull request with a YAML patch and a root-cause breakdown. Every single AI commit needs to pass through automated linting, security scans, and Open Policy Agent (OPA) checks before it gets anywhere near the main branch.
-
What you must block: Never let an AI auto-merge massive architectural changes, like tweaking IAM roles or database connection strings. Also, put hard limits on retry loops. If an AI keeps trying to force a broken deployment, it’s going to chew through our API quotas fast.
2. Security (DevSecOps): Locking Down the Attack Surface
Agents use protocols like the Model Context Protocol (MCP) to talk to internal tools, but it opens our attack surface wide open. A hijacked URL or a weirdly formatted document can trick an AI into running rogue shell commands.
-
What you should automate: Move our access control completely outside the AI’s “brain.” Use OPA sidecars to intercept and validate every single API call an agent attempts. Keep an automated AI Bill of Materials (AI-BOM) to track every agent and tool in play so you don’t end up with shadow AI running wild across network.
-
What you must block: Stop handing out permanent API keys to AI agents. Use tightly scoped, short-lived tokens instead. And never automate authorization for destructive actions like tearing down infrastructure. Make a human pass a multi-factor authentication (MFA) challenge first.
3. Finance (FinOps): Keeping the Cloud Bill in Check
Normal cloud compute costs are fairly predictable. AI agent costs are not. They fluctuate wildly depending on token usage and how many reasoning loops the agent gets stuck in. An unchecked agent can burn a month’s budget in a single week.
-
What you should automate: Set up automated pipelines to ingest and normalise our billing data (the FOCUS 1.3 spec is great for this). Configure real-time anomaly detection so you get a Slack/Team alert the second an agent starts eating up too many tokens. You should also automate shutting down idle AI environments ( if you are hosting AI ) after hours.
-
What you must block: Don’t let an AI sign 3-year Reserved Instance (RI) contracts or buy GPU capacity. Financial strategy needs human context—cash flow constraints, vendor leverage, and future architecture changes that an AI simply doesn’t know about.
Flipping the Script: AI-in-the-Loop
The old way of handling automation was “Human-in-the-Loop,” where the AI pauses and asks for permission every five seconds. As agents get faster, this creates intense approval fatigue. Engineers get sick of the pop-ups and start rubber-stamping everything, completely defeating the purpose of having a security gate in the first place.
The fix is flipping the model to “AI-in-the-Loop” (AI2L). The human acts as the strategic orchestrator.
Instead of pinging you for every minor API call, the AI batches its work. It hands you a complete strategy: “Here’s the code fix, the security scan, and the estimated cost.” The AI handles the boring stuff autonomously within strict OPA guardrails and only escalates to you when something crosses a high-risk threshold.
Agentic DevSecFinOps brings serious speed to the table. But speed without guardrails is just a faster way to crash. By locking AI execution behind rigid, rule-based deterministic systems and keeping humans in charge of the high-stakes decisions, engineering teams can actually use autonomous infrastructure without losing sleep ( read on-call ) over it.
