Automating agentic AI Well-Architected reviews with the AWS Agentic AI Lens

In regulated industries, there’s no room for a move fast and break things approach, especially when AI agents are handling sensitive customer communications data. In this post, we show how Smarsh, a leading archiving and compliance platform for communications intelligence, used the AWS Well-Architected Framework’s Agentic AI Lens to conduct a rigorous, evidence-based review of its first agentic workload. The Agentic AI Lens is available today as prescriptive guidance, though it is not yet selectable as a lens in the AWS Well-Architected Framework.

To operationalize the review, Smarsh built a custom skill for Claude, Anthropic’s AI assistant. The skill assessed the workload against all 150 best practices in the lens, pulling evidence directly from the workload’s GitHub repository and producing a comprehensive review workbook in hours rather than days. In this post, we walk through the skill’s architecture, the assessment methodology, how the team validated the skill itself, and how the resulting findings strengthened Smarsh’s engagement with its AWS account team.

About Smarsh

Smarsh is a recognized global leader in electronic communications archiving and digital communications governance. For over 20 years, the multinational software as a service (SaaS) company has enabled more than 7,000 organizations, from small businesses to large multinational financial institutions, to capture, retain, monitor, and analyze communications across 100+ channels, including email, mobile, voice, video, chat, and AI assistants, at scale. The company’s enterprise archive business runs on Amazon Web Services (AWS), and its next generation of AI capabilities are being built on Amazon Bedrock AgentCore, a platform to build, connect, and optimize agents at scale, with any framework or model. Smarsh uses AgentCore’s runtime, gateway, registry, and identity capabilities to power these solutions. A 2025 AWS Partner of the Year Award winner, Smarsh holds a deep partnership with AWS.

Prerequisites

To follow along with the approach described in this post, you need the <a href="https://myappsplus.com/reddit-now-lets-you-choose-between-for-you-and-following-feeds/” title=”Reddit now lets you choose between For You and Following feeds”>following:

• An AWS account with access to Amazon Bedrock and Amazon Bedrock AgentCore.
• Familiarity with the AWS Well-Architected Framework and its lens model.
• A GitHub repository containing your agentic workload’s source code, infrastructure definitions, and documentation.
• Access to Claude (Anthropic’s AI assistant) with the ability to install custom skills.

Smarsh operates in a highly regulated industry where compliance and security are paramount. Every agentic solution must pass a Well-Architected review before it reaches an end customer. So when a Smarsh architect set out to review the company’s first multi-agent workload, the natural choice was the AWS Well-Architected Framework’s Agentic AI Lens. That workload is an intelligent agent review system for compliance alerts built on the runtime, gateway, registry, and identity capabilities of Amazon Bedrock AgentCore.

The Serverless Applications Lens and other lenses can be added directly in the AWS Well-Architected Tool. The Agentic AI Lens, by contrast, existed only as published documentation: 150 best practices across 41 questions and six pillars, each with a risk level and a desired outcome, but with no way to run the review natively on the console. That left two apparent options: a multi-day manual effort in which every reviewer structured the assessment differently, or waiting for console support. Smarsh chose a third: make the lens executable, in a way every future workload could reuse.

Solution architecture: The lens as data, the reviewer as a skill

Smarsh packaged the review process as a Claude skill: a versioned bundle of instructions, reference data, and executable scripts that Claude discovers through a frontmatter description and loads on demand. Progressive disclosure keeps the token cost near zero until a review is actually requested. The skill has three components:

• lens_data.json – All 150 best practices from the published lens documentation, extracted into a structured JSON dataset: best-practice ID, pillar, review question, title, risk level (90 High, 59 Medium, 1 Low), and desired outcome. Risk levels were cross-validated against the lens appendix to catch extraction errors. This dataset is the single source of truth, and the skill is instructed never to invent or renumber items.
• assessment-guide.md – A five-value status rubric (Compliant, Partially Compliant, Non-Compliant, Not Applicable, and Not Reviewed), an evidence-gathering checklist, per-pillar signals to look for, and a set of honesty rules. The checklist covers README and architecture docs, decision records, infrastructure as code, continuous integration and continuous delivery (CI/CD) workflows, and test and evaluation suites. Every status must cite evidence found in the repository, items with no signal are marked Not Reviewed rather than guessed, and the same underlying fact must produce a consistent status wherever it appears across pillars.
• build_report.py – A deterministic Python script (openpyxl) that renders the model’s assessment into a formatted workbook. The generator validates its input, so unknown best-practice IDs or invalid statuses abort the build and hallucinated items cannot silently ship. The output includes per-item status with data-validation dropdowns and conditional formatting. It also provides columns for cited evidence, gaps, and remediation, plus a summary sheet whose pillar counts and high-risk exposure tracker are live COUNTIF formulas rather than hardcoded numbers.

The assessment contract between the reasoning and the rendering is in a single JSON document:

{
  "AGENTSEC08-BP01": {
    "status": "Partially Compliant",
    "evidence": "Two-stage input validation (JSON Schema + Pydantic); 5 MB payload cap; typed prompts.",
    "gap": "No prompt-injection screening on message content.",
    "remediation": "Add injection screening; treat communication text as untrusted data in prompts."
  }
}

Running a review is a one-line request: point Claude at a repository and ask for a Well-Architected review. From there, the skill drives the following workflow:

1. Claude loads the skill and reads the frontmatter description to understand the review scope and methodology.
2. The skill pulls the lens_data.json dataset containing all 150 best practices across six pillars.
3. Claude gathers evidence read-only from the target repository: README, architecture docs, infrastructure-as-code, CI/CD workflows, and test suites.
4. The skill assesses all 150 items against the gathered evidence, applying the five-value rubric and honesty rules from assessment-guide.md.
5. Claude calls build_report.py to generate the formatted workbook, validating row counts and flagging blank statuses.
6. The skill returns an executive summary of the top high-risk gaps.
7. Human reviewers validate findings, feed a remediation backlog, and surface open system questions for a focused AWS deep dive.

Figure 1: From published lens documentation to an evidence-backed review workbook

Testing the tester: Evaluating the skill before trusting it

Smarsh evaluated the skill before relying on its output. The team built two synthetic fixture repositories: a moderately mature LangGraph-style workload and a deliberately rough proof of concept, complete with hardcoded credentials and a runaway-cost incident in its history. The team ran reviews both with and without the skill. Each run was graded against eight verifiable expectations per fixture: all 150 lens items present, no blank statuses, evidence cited for every definitive status, honest calibration, and fixture-specific findings such as the hardcoded credential being flagged. With the skill, runs passed 16 of 16 expectations. Baseline runs without it passed only 8 of 16, producing ad-hoc 18–39 item checklists with no lens traceability. The eval set ships inside the skill, so regression testing continues as the skill evolves.

What the first review found

Against the real workload, the skill read the repository’s architecture and authentication documentation, AWS Cloud Development Kit (AWS CDK) infrastructure definitions, GitHub Actions workflows, registry and gateway design records, and test suites. The skill then assessed all 150 items in hours. The workbook showed a credible mix of strengths and gaps. Strengths included per-agent AWS Identity and Access Management (IAM) execution roles scoped to named runtime Amazon Resource Names (ARNs) and SigV4-signed Model Context Protocol (MCP) calls through AgentCore Gateway, a capability of Amazon Bedrock AgentCore. The workbook also recorded schema-validated inputs with typed error contracts, exported decision audit trails, and continuous quality monitoring through AgentCore online evaluations. High-risk gaps included the gateway ingress authorizer running with authentication disabled under a documented risk acceptance, no prompt-injection screening on bad actor-influenced message content, rate limiting recorded as a TODO, and no automated cost controls. Each gap was tied to the lens item, the evidence, and a recommended remediation.

Next, the Smarsh architect brought in the engineering team. Together they triaged the findings, validating statuses against what the team knew, accepting or challenging results right in the workbook, and ranking the remediation backlog. Engineering’s input reshaped the review: several items collapsed into one anomaly-detection epic, and a single finding was upgraded once the team pointed to evidence the repository had overlooked. Dependency scanning was already running on every pull request.

Sharper questions, better AWS engagement

Triage surfaced a class of gaps no repository can answer: system-semantics questions about the AWS services themselves. Rather than booking a general architecture review, the team built a focused agenda for its AWS Solutions Architects, mapping each question to the checklist item it would close. The agenda covered:

• The durability and consistency of AgentCore runtime, a capability of Amazon Bedrock AgentCore, as a session checkpoint store.
• The retry and idempotency semantics of the InvokeAgentRuntime API.
• Break-glass patterns for halting agent invocations.
• Amazon Bedrock Guardrails patterns for screening untrusted content.
• Penetration-testing rules of engagement for Amazon Bedrock AgentCore.
• Amazon GuardDuty and AWS CloudTrail coverage for agentic-runtime threat detection.

The deep dive became a working session with a definition of done.

Why AWS

The security and compliance posture of Amazon Bedrock and Amazon Bedrock AgentCore, spanning runtime, gateway, registry, identity, and evaluation capabilities, meets the bar Smarsh’s regulated customers expect. The AWS Well-Architected Framework already anchors how Smarsh evaluates workloads. Extending it to agentic AI through the published lens, and pairing automated reviews with AWS Solutions Architect expertise to close the hard gaps, is a natural continuation of how the two teams work together.

Results and expected outcomes

These numbers reflect Smarsh’s first internal uses and the outcomes the team is tracking toward.

1. Review effort – A complete 150-item, evidence-cited draft in hours instead of a multi-day manual effort, with human time refocused on validation and prioritization.
2. Comparable reviews – Every workload produces the same standardized workbook, which Smarsh expects to translate into faster security approvals and improved time-to-market for agentic solutions.
3. Sharper use of AWS expertise – Instead of broad discussions, AWS engagements became itemized working sessions, each mapped to a specific best-practice gap.

What’s next

1. Publish the skill internally as the standard evaluation tool for Smarsh agentic workloads. It has been contributed to the company’s internal AI-SDLC plugin marketplace and has already been run against a second, architecturally different agentic repository.
2. Bake reviews into the software development lifecycle so well-architected checks run continuously during development, not as a one-time gate at the end.
3. Enforce governance by making the review workbook required evidence in Smarsh’s project milestone and gate process before agentic workloads reach production.
4. Expand the lens-as-data pattern to other AWS Well-Architected lenses and additional repositories, eventually covering the broader application portfolio.

Conclusion

The gap Smarsh hit, prescriptive guidance published faster than console tooling can absorb it, is one many customers will meet as agentic AI best practices evolve. The answer generalizes: treat the lens as data, encode the reviewer’s discipline as a skill with enforced honesty rules, validate the tool with its own evaluation suite, keep humans on judgment, and let findings drive precise conversations with AWS. A review that once implied days of manual effort is now a repeatable capability the whole organization can run, before, not after, an agentic workload meets a regulated customer.

• Explore the AWS Well-Architected Framework and the Agentic AI Lens documentation.
• See the Amazon Bedrock AgentCore documentation for building agentic workloads.
• Contact your AWS account team to schedule a Well-Architected review of your own agentic workloads.

We’d love to hear how you’re approaching Well-Architected reviews for your own agentic workloads. Share your experience in the comments, or reach out to your AWS account team to get started.