Skip to main content

When an agent becomes the front door to your support organization, it stops being an experiment. It is production infrastructure with a customer-facing SLA, and it fails in ways your existing runbooks have never seen.

Most teams get the build right. They ship the agent, wire up retrieval, add guardrails, and watch the traces. Then someone asks what happens if the region hosting it goes away, and the room gets quiet. The answer is usually a mix of “we have Terraform” and “we would rebuild it,” which is not a recovery plan with a number attached to it.

We are going to walk you through a fictitious, but realistic scenario with Digitata. Digitata is a B2C service provider and they want to jump on the agentic wave by adding a virtual customer service agent that is powered by Amazon Bedrock. 

When you think of building an agentic application, you need to consider resilience for the agent equal to the data and infrastructure. To put this in context, we’ll walk through a fictional case with the Digitata customer service agent.

The Digitata customer service agent

Meet Digitata, an enterprise B2C company with an active customer service team.  Many of the customer service requests are product questions. In order to free time for human agents, Digitata has built an agentic interface for customers to address these queries. The agent will replace the first tier of the ticket queue, and greatly improve resolution time.


Digitata built their agentic application using Bedrock AgentCore along with other supporting Bedrock services, it’s been tested in development and QA environments, and we are now ready to ship it to production.

How the components relate:

 

AgentCore Runtime Hosts the agent itself. The orchestration code ships as a container image in Amazon ECR, and the runtime executes it under a scoped execution role.
Bedrock Knowledge Base Grounds answers in Digitata’s product documentation. Docs land in an S3 data source; the resulting embeddings live in an OpenSearch Serverless vector collection, and answers cite the source page.
Bedrock Guardrails Keeps the agent inside its job. Denied topics stop it answering general questions, and content filters screen what comes back before a customer sees it.
AgentCore Memory Carries context across sessions, so a customer who returns tomorrow does not restate their account, their plan, and the problem they were already halfway through.
Supporting AWS resources IAM execution roles, VPC and security group configuration, secrets, and the CloudWatch alarms and traces that make the whole thing observable.

The production readiness checklist

With their agent built in a sandbox with all of the needed Amazon Bedrock resources, the next step for the Digitata team was getting it to production.  This process entails much more than a demo. The Digitata team created a production-ready checklist that the agents must pass to meet production-readiness:

Behavior and Quality

  • Guardrail policies pinned to a published version, with adversarial prompts running in CI
  • Scope and refusal evals gating every deploy
  • A golden set of documentation questions re-run after each ingestion

Operations

  • Alarms on failed ingestion, guardrail intervention rate, tool errors, and model throttling
  • Per-session traces, plus sampled human review of real conversations
  • Per-session token ceilings and a monthly budget alarm

Disaster Recovery (1-hour RTO)

  • RTO and RPO agreed with the business and written down
  • Every component in scope: runtime image, agent configuration, guardrails, knowledge base, vector index, memory, roles, network
  • Recovery points in a separate account that production credentials cannot reach or delete
  • A drill on a schedule that produces a working agent answering real questions

Building the DR plan by hand

When they got to the final requirement of the production-ready checklist, Digitata hit a roadblock: they hadn’t incorporated their Amazon Bedrock services into their DR plan. “It is all in Terraform” is a good starting position, but an incomplete one. Terraform recreates resources; it does not carry the data inside them, and several of these components hold state that has no export button. Their first instinct was to build the disaster recovery plan by hand.

Digitata began to outline their recovery component by component:

Component What recovery requires
Runtime container image The image sits in one region’s ECR registry. A runtime in the recovery region cannot pull it. You need cross-region replication and a guarantee that the exact tag production was running is present, not just the latest build.
Agent configuration Model selection, system prompt, tool and MCP wiring, and session settings live in the control plane. Anything a developer changed in the console and never backported to code does not exist in your recovery environment.
Guardrails Guardrails are regional, versioned resources referenced by ID. Recreate them in the recovery region and the IDs differ, so every reference has to be resolved at deploy time rather than hardcoded.
Knowledge base & S3 source S3 replication handles the documents. The knowledge base resource, its embedding model configuration, and its data source binding all have to be recreated and re-associated.
OpenSearch Serverless vectors Vectors do not travel with the S3 bucket. Either you re-ingest the whole corpus in the recovery region, which takes as long as it takes and is usually the long pole in your RTO, or you run and pay for a second continuously synced collection.
AgentCore Memory Memory is the accumulated value of every customer conversation, and there is no bucket you can point a nightly job at. Getting it out means building and owning a streaming pipeline.
IAM, network, secrets Role ARNs, VPC IDs, subnet IDs, and secret ARNs are account and region specific. Every one has to be parameterized, and a single hardcoded ARN fails the deploy at the worst possible moment.

Building the recovery path yourself

Digitata has a strong infrastructure and operations team; and they weren’t about to take any shortcuts on an initiative that was client-facing. The identified five workstreams to deliver a tested one-hour recovery.

  1. Refactor the IaC to be account and region agnostic. Every account ID, region, ARN, and resource name becomes a variable. Provider aliases, separate state, and a second set of tfvars. Then the harder part: keeping the recovery configuration honest as the application changes weekly.
  2. Build a memory backup pipeline. Stream memory events to Kinesis, and replicate them to a copy of the memory in the recovery environment. This is a new production data pipeline, with its own monitoring, its own failure modes, and its own on-call surface.
  3. Replicate the documentation and the image. S3 cross-region replication for the knowledge base data source, ECR replication rules for the runtime image, and a decision on the vector store: pay to keep a warm collection synced, or accept a re-ingestion window inside your RTO.
  4. Write and script the recovery runbook. Ordering matters: knowledge base before agent, guardrail version resolved before the runtime starts, memory reloaded before traffic arrives. The parts that cannot be automated become manual steps, and manual steps are where an hour turns into an afternoon.
  5. Test it, repeatedly. An untested recovery plan is a hypothesis. Each drill means standing up the environment, validating that the agent answers grounded questions with memory intact, and tearing it down without disturbing production.

Digitata realized that the initial build is finite work, but the ongoing cost erodes plans: every new tool the agent gains, every guardrail policy change, every VPC adjustment has to land in the recovery configuration too, or your tested plan quietly stops matching production. When the disaster recovery process doesn’t match, Digitata’s ability to respond to an outage event would be broken.

Ransomware Protection

Digitata’s team is ready for an outage. They built up their DR plan that accounts for changes to the agents and they are ready in case there are any disruptions to the underlying resources that support not only their agent, but also their cloud application.  However, there is one aspect of recovery that they missed: a cyber attack.

Being cyber resilient is similar to disaster resilience, but there is some nuance. Replication is designed to propagate changes faithfully, which means it propagates corrupted container images or deleted memories just as faithfully.

Recovering from a cyber event needs something replication cannot give you: historical recovery points an attacker with production credentials cannot reach, held in an isolated account, plus somewhere clean to validate that the threat is gone before anything serves traffic again.

While considering Cyber resilience, Digitata realized they had a big challenge keeping up with the pace of change with their new agents while meeting the resilience requirements set forth by the risk and compliance team.

The same result, in minutes

Digitata turned to Arpio because only Arpio protects Amazon Bedrock, including AgentCore runtimes and memory, knowledge bases, and guardrails, alongside the rest of the AWS resources the application depends on. There is no pipeline to build and no recovery configuration to maintain in parallel with production. With Arpio, resilience can be set up in four steps.

Step 1 — Point Arpio at the source environment

Connect the production account and the recovery account, then scan. Arpio inventories what is running, including the agent runtime, its guardrails, its knowledge base and vector store, and its memory.

Step 2 — Let dependency detection build the protection scope

Select the application and Arpio follows its dependencies outward: the ECR image the runtime pulls, the S3 data source behind the knowledge base, the IAM roles, the network.

Step 3 — Continuous, isolated recovery points

Arpio syncs the whole scope into the recovery account on a schedule, supporting a 15-minute RPO for this application. Recovery points are held outside production’s reach, so a compromise of the production account does not take the backups with it.

Step 4 — Test on demand, then recover in one click

Run a test into an isolated sandbox whenever you want, without touching production, and confirm the agent answers grounded questions with its memory intact. When it counts, the same automation runs for real: an RTO of 15 minutes for this application, well inside the one-hour target.

Digitata’s Result

After a side-by-side comparison, Digitata’s I&O team realized that providing a resilient agentic application added a lot of unforeseen scope to ensure it was ready to meet the compliance and risk teams’ resilience requirements. They chose Arpio because the platform not only understood their full AWS environment, but also understood their agents as well.  This ensures that their application can remain resilient in the face of outages, disasters, and even a cyber attack.

 

Built by hand With Arpio
Time to first protection Weeks of platform work across five workstreams Minutes, in the console
Agent memory A streaming pipeline you build, run, and monitor Covered in scope
Vector store Re-ingest during the outage, or pay for a warm second collection Recovered with the application
Keeping up with change Manual, every time the application changes Rescanned continuously
Testing A scheduled project with production risk On demand, isolated from production
Ransomware Replication propagates the damage Historical points in an isolated account

 

See it against your own agent

Bring the architecture you are about to ship and we will scan it, show you the dependencies, and run a recovery test with you.

Request a Demo »