Designing a multi-AZ architecture is the easy half of resilience. The hard half is proving it works: that the Auto Scaling group really replaces the instances you lose, that the database really fails over inside your recovery time objective, and that the alarm you wrote two quarters ago still fires. Most teams discover the answer during a real outage, which is the most expensive time to find out.
AWS gives DevOps engineers two managed services to find out earlier. AWS Fault Injection Service (FIS) runs controlled chaos-engineering experiments — it actually breaks things, on purpose, with guardrails. AWS Resilience Hub analyzes an application’s architecture against recovery targets you define and tells you, before anything breaks, whether you can meet them. Together they close the loop between “we designed for resilience” and “we verified it.”
For the AWS Certified DevOps Engineer - Professional (DOP-C02) exam, these services sit behind scenario phrases like “validate that the workload meets its RTO,” “test the failover process without risking an uncontrolled outage,” and “continuously assess resilience as the architecture changes.” This guide covers how both services work, the configuration details the exam leans on, and how to choose between them.
Where This Fits on the DOP-C02 Blueprint
The Resilient Cloud Solutions domain carries roughly 15% of the DOP-C02 exam and asks you to implement highly available solutions, scalable solutions, and automated recovery processes that meet RTO and RPO requirements. The Resilient Cloud Solutions guide covers the design side: the four disaster recovery strategies, Auto Scaling, load balancing, Route 53 failover, and resilient data stores.
This article is the verification side. Think of the split this way:
| Question | Service | What it does |
|---|---|---|
| ”Can this architecture meet our RTO/RPO?” | AWS Resilience Hub | Static assessment of the architecture against a resiliency policy |
| ”Does it actually recover when something fails?” | AWS Fault Injection Service | Injects real faults and lets you observe the real response |
| ”Did the alarm and runbook work?” | Both | Resilience Hub recommends alarms, SOPs and tests; FIS exercises them |
Both services appear in the DOP-C02 in-scope services list, so expect them as correct answers and as distractors.
Chaos Engineering in One Paragraph
Chaos engineering is the discipline of running controlled experiments on a system to build confidence that it withstands turbulent conditions. The method is always the same: define the steady state (a measurable signal of healthy behaviour, such as p99 latency or successful order rate), form a hypothesis (“if we lose one AZ, the order rate stays above 99%”), inject a fault, observe, and then fix whatever the experiment exposed. The two ideas the exam cares about are that the experiment is controlled — a limited blast radius and an automatic abort — and that it is measurable, which on AWS means CloudWatch metrics and alarms.
AWS Fault Injection Service: The Building Blocks
FIS is a fully managed service for running fault injection experiments on AWS workloads. Everything revolves around the experiment template, a reusable definition with four core parts.
Actions
An action is the fault you inject. Actions are identified by an ID in the form aws:<service>:<action>. A representative sample:
| Action ID | What it does |
|---|---|
aws:ec2:stop-instances / aws:ec2:terminate-instances | Stops or terminates target EC2 instances |
aws:ec2:send-spot-instance-interruptions | Sends a Spot interruption notice to target Spot Instances |
aws:ecs:stop-task | Stops target ECS tasks |
aws:eks:terminate-nodegroup-instances | Terminates a percentage of instances in an EKS managed node group |
aws:rds:failover-db-cluster | Forces an Aurora DB cluster failover |
aws:rds:reboot-db-instances | Reboots RDS instances, optionally forcing a Multi-AZ failover |
aws:network:disrupt-connectivity | Blocks traffic to and from target subnets |
aws:ebs:pause-volume-io | Pauses I/O on target EBS volumes |
aws:fis:inject-api-throttle-error | Makes AWS API calls from a target IAM role return throttling errors |
aws:ssm:send-command | Runs an SSM document on target instances (CPU stress, latency, and more) |
aws:fis:wait | Does nothing for a set duration — used to space out other actions |
Actions run in parallel by default. To sequence them, give an action a startAfter list naming the actions that must finish first. That is how you model a realistic cascading failure: add latency, wait, then terminate instances.
Many actions take a duration parameter in ISO 8601 format (for example PT5M for five minutes). For actions like network disruption or paused I/O, FIS reverts the fault automatically when the duration ends or the experiment stops. Actions such as terminating an instance are, of course, not reversible — your architecture is supposed to handle the recovery.
Targets
A target is the set of resources an action applies to. You identify resources by:
- Resource IDs (explicit ARNs), or
- Resource tags (for example
env=staging), optionally narrowed with - Filters on resource attributes, such as
Placement.AvailabilityZoneorState.Name.
You then choose a selection mode that controls blast radius:
| Selection mode | Meaning |
|---|---|
ALL | Every matching resource |
COUNT(n) | n resources chosen at random from the matches |
PERCENT(n) | n percent of the matching resources, chosen at random |
Tag-based targeting plus PERCENT or COUNT is the standard answer whenever a scenario says “limit the experiment to a subset of the fleet” or “only affect non-production instances.”
Stop Conditions
A stop condition is the safety mechanism, and it is the single most testable FIS concept. A stop condition is a CloudWatch alarm: if the alarm enters the ALARM state while the experiment is running, FIS stops the experiment automatically. If the alarm is already in ALARM when you start, the experiment does not run.
You typically attach an alarm on your steady-state metric — error rate, latency, or a composite alarm that aggregates several signals. When an exam question asks how to “automatically halt the experiment if customer impact exceeds a threshold,” the answer is a stop condition backed by a CloudWatch alarm, not a Lambda function polling metrics.
IAM Role
FIS performs actions by assuming an experiment role you specify in the template. The role trusts the fis.amazonaws.com service principal and needs permissions for exactly the actions in the template — ec2:TerminateInstances for instance termination, rds:FailoverDBCluster for Aurora failover, and so on. Scoping that role tightly, ideally with tag-based conditions, is a second layer of blast-radius control on top of target selection.
A Complete Experiment Template
Here is a template that terminates half of the running, tagged web instances in a single Availability Zone and aborts if the 5xx alarm fires:
{
"description": "Terminate 50% of web instances in us-east-1a",
"targets": {
"webInstances": {
"resourceType": "aws:ec2:instance",
"resourceTags": { "app": "web", "env": "staging" },
"filters": [
{ "path": "Placement.AvailabilityZone", "values": ["us-east-1a"] },
{ "path": "State.Name", "values": ["running"] }
],
"selectionMode": "PERCENT(50)"
}
},
"actions": {
"terminateInstances": {
"actionId": "aws:ec2:terminate-instances",
"targets": { "Instances": "webInstances" }
}
},
"stopConditions": [
{
"source": "aws:cloudwatch:alarm",
"value": "arn:aws:cloudwatch:us-east-1:111122223333:alarm:web-5xx-high"
}
],
"roleArn": "arn:aws:iam::111122223333:role/fis-experiment-role",
"tags": { "Name": "az-a-web-termination" }
}Create the template and run it with the AWS CLI:
# Create the reusable template
aws fis create-experiment-template --cli-input-json file://template.json
# Start an experiment from it
aws fis start-experiment --experiment-template-id EXT1a2b3c4d5e6f7
# Check progress and per-action state
aws fis get-experiment --id EXP9z8y7x6w5v4u3
# Manually abort if you need to
aws fis stop-experiment --id EXP9z8y7x6w5v4u3An experiment moves through states such as pending, initiating, running, and then completed, stopped (a stop condition triggered or someone stopped it), or failed. Experiment templates are also a CloudFormation resource type (AWS::FIS::ExperimentTemplate), so they belong in version control next to the infrastructure they test.
Injecting Faults Inside the Instance with SSM
API-level faults (terminate, stop, fail over) do not cover everything. To simulate a noisy neighbour or a slow dependency you need a fault inside the operating system. FIS does this through the aws:ssm:send-command action, which runs a Systems Manager document on the targets. AWS publishes pre-built documents for the common cases:
| SSM document | Fault |
|---|---|
AWSFIS-Run-CPU-Stress | CPU stress |
AWSFIS-Run-Memory-Stress | Memory stress |
AWSFIS-Run-IO-Stress | Disk I/O stress |
AWSFIS-Run-Disk-Fill | Fills the root volume to a target percentage |
AWSFIS-Run-Network-Latency | Adds latency on a network interface |
AWSFIS-Run-Network-Packet-Loss | Drops a percentage of packets |
AWSFIS-Run-Network-Blackhole-Port | Drops traffic for a protocol and port |
AWSFIS-Run-Kill-Process | Kills a named process |
The prerequisite is the same as for any Run Command use: the instances must be managed nodes with SSM Agent running and an instance profile that allows Systems Manager. If that sounds unfamiliar, the Systems Manager operations guide covers the managed-node requirements. FIS can also run a full Automation runbook via aws:ssm:start-automation-execution when a fault needs multiple orchestrated steps.
Scenarios, Multi-Account Experiments, and Logging
Three more FIS features round out exam-level knowledge:
- Scenario library. FIS ships pre-built, multi-action scenarios you can copy into a template. The headline ones are AZ Availability: Power Interruption, which simulates the loss of a whole Availability Zone across compute, networking, and databases, and Cross-Region: Connectivity, which simulates a Region becoming unreachable. When a question asks for the least-effort way to test an AZ-failure hypothesis, a scenario from the library beats hand-assembling a dozen actions.
- Multi-account experiments. A single experiment can target resources in several accounts. You run it from an orchestrator account and add target account configurations, each with a role that FIS assumes in that account. This matters for the multi-account organizations DOP-C02 scenarios love.
- Experiment logging. Templates can send experiment logs to CloudWatch Logs or Amazon S3, giving you a timestamped record of each action to correlate with dashboards and for audit evidence.
FIS activity is also recorded in CloudTrail, and experiment state changes are emitted to EventBridge, so you can notify a channel or trigger automation when an experiment starts, stops, or fails.
AWS Resilience Hub: Assess Before You Break
Where FIS is empirical, AWS Resilience Hub is analytical. It is a central place to define, validate, and track the resilience of applications against explicit recovery targets. The workflow has four steps.
1. Describe the Application
You tell Resilience Hub what the application is made of by pointing it at one or more input sources: CloudFormation stacks, Terraform state files, AWS Resource Groups, AppRegistry applications, or Amazon EKS clusters. Resilience Hub discovers the resources and groups them into application components (compute, database, storage, and so on).
2. Attach a Resiliency Policy
A resiliency policy sets the target RTO and RPO for each disruption type:
| Disruption type | Example failure |
|---|---|
| Application | A bad deployment or software/configuration fault |
| Infrastructure | Loss of an instance or other hardware |
| Availability Zone | An entire AZ becomes unavailable |
| Region (optional) | An entire Region becomes unavailable |
Policies also carry a tier label (for example Mission Critical, Critical, Important, Core Services, Non-Critical) so you can standardize targets across a portfolio. If RTO and RPO are hazy, revisit the definitions in the resilient solutions guide — every Resilience Hub question assumes you know them cold.
3. Run an Assessment
Publishing the application version and running an assessment makes Resilience Hub evaluate each component’s configuration and estimate the achievable RTO and RPO per disruption type. The outcome is a compliance status — the policy is either met or breached — plus a resiliency score from 0 to 100 that reflects how closely the application follows the recommendations.
A typical finding reads like: “The RDS instance is single-AZ; estimated AZ-disruption RTO is hours against a 30-minute target — policy breached.”
4. Act on the Recommendations
Assessments produce two families of recommendations:
- Resiliency recommendations — architecture changes per component, offered as alternatives optimized for the lowest AZ RTO/RPO, the lowest cost, or the fewest changes.
- Operational recommendations — the things you need around the architecture: CloudWatch alarms to detect failure, standard operating procedures (SOPs) implemented as Systems Manager Automation documents, and tests implemented as FIS experiments.
Resilience Hub can generate CloudFormation templates for those alarms, SOPs, and experiments, so the recommendations become deployable infrastructure rather than a to-do list. Implementing them raises the resiliency score.
Keeping It Continuous
Two features turn a one-off review into ongoing governance:
- Scheduled assessments re-run the assessment daily.
- Drift detection flags when an application that previously met its policy no longer does, or when the underlying resources have changed, and can notify you through Amazon SNS.
That combination is the answer to scenarios asking how to “be alerted when a change causes the application to stop meeting its recovery objectives.”
FIS vs Resilience Hub: A Decision Cheat Sheet
| Scenario cue | Answer |
|---|---|
| ”Simulate an AZ outage and observe recovery” | FIS (AZ Availability: Power Interruption scenario) |
| “Inject CPU stress or network latency on instances” | FIS aws:ssm:send-command with an AWSFIS-Run-* document |
| ”Automatically abort the test if errors spike” | FIS stop condition on a CloudWatch alarm |
| ”Limit the test to 10% of tagged instances” | FIS target with resource tags and PERCENT(10) |
| ”Determine whether the architecture can meet a 15-minute RTO” | Resilience Hub assessment against a resiliency policy |
| ”Get recommended alarms, runbooks, and tests for an application” | Resilience Hub operational recommendations |
| ”Detect when a change breaks RTO/RPO compliance” | Resilience Hub scheduled assessments and drift detection |
| ”Test how the app handles throttled AWS API calls” | FIS aws:fis:inject-api-throttle-error |
| ”Force an Aurora failover to measure real recovery time” | FIS aws:rds:failover-db-cluster |
A useful mental shortcut: Resilience Hub estimates, FIS measures. A complete answer to “prove we meet our objectives” usually involves both — Resilience Hub to find the gaps and recommend experiments, FIS to run them.
Wiring Resilience Checks into CI/CD
DOP-C02 rewards automation, and both services are API-driven, so they fit into a delivery pipeline:
- Deploy the change to a staging environment with your usual pipeline (see the CI/CD guide).
- Assess: a pipeline step (a CodeBuild job or a Step Functions state machine) imports the updated resources into Resilience Hub, publishes a new application version, and starts an assessment. If the policy is breached, fail the stage.
- Experiment: a second step starts an FIS experiment from a versioned template and waits for it to finish. A
stoppedorfailedstate fails the stage. - Promote only if both gates pass.
# Gate 1: assess the new application version
aws resiliencehub start-app-assessment \
--app-version-arn "$APP_ARN" \
--assessment-name "pipeline-$CODEBUILD_BUILD_NUMBER"
# Gate 2: run the experiment and capture its ID
EXP_ID=$(aws fis start-experiment \
--experiment-template-id "$TEMPLATE_ID" \
--query 'experiment.id' --output text)
aws fis get-experiment --id "$EXP_ID" --query 'experiment.state.status'Pair this with safe rollout mechanics from the deployment strategies guide and the alarm-driven rollbacks described in the monitoring guide, and resilience becomes a tested property of every release rather than an annual exercise.
Running Experiments Safely
Whether in a lab or in production, the same practices apply:
- Start in non-production, then graduate to production with a small blast radius.
- Always define a stop condition. A template with no stop condition has no automatic brake.
- Scope with tags and
PERCENT/COUNT, neverALLon a first run. - Scope the experiment role to the specific actions and, where possible, tagged resources.
- Watch the steady-state dashboard during the run; the experiment’s value is in what you observe.
- Turn findings into fixes — a missing alarm, a slow health check, a runbook step that no longer works — and re-run to confirm. The incident response and auto-remediation guide covers building the remediation side.
Common Exam Traps
- Confusing the two services. Resilience Hub does not inject faults; FIS does not calculate RTO/RPO. Options that swap these roles are wrong.
- Building the abort yourself. A Lambda function that polls metrics and calls
StopExperimentworks, but a CloudWatch alarm stop condition is the managed, lower-effort answer. - Forgetting SSM prerequisites. In-instance faults need SSM Agent and an instance profile; without managed nodes,
aws:ssm:send-commandhas nothing to run on. - Assuming FIS only does EC2. It covers ECS, EKS, RDS and Aurora, EBS, networking, Spot interruptions, and AWS API error injection.
- Treating Resilience Hub as a backup service. It assesses and recommends; AWS Backup takes the backups, and failover mechanics live in Route 53, RDS, and the rest of your architecture.
- One-time assessment. If the scenario says “continuously” or “as the architecture changes,” look for scheduled assessments and drift detection.
Practice on Realistic DOP-C02 Scenarios
Resilience questions at the professional level are rarely about definitions. They describe an architecture, a recovery target, and a constraint — least operational overhead, no customer impact, multi-account — and ask you to pick the verification approach. Recognizing the cues quickly takes repetition against scenario-style questions with explanations for why each distractor fails.
Sailor.sh’s AWS Certified DevOps Engineer - Professional (DOP-C02) mock exam bundle offers exam-weighted practice across all six domains, including Resilient Cloud Solutions, with explanations that focus on that “why this and not that” reasoning. Use it alongside the DOP-C02 exam guide for the blueprint, the DOP-C02 study plan to sequence your preparation, and the practice questions walkthrough for a first taste of the format.
Frequently Asked Questions
What is AWS Fault Injection Service used for?
AWS Fault Injection Service (FIS) is a managed service for running chaos-engineering experiments on AWS workloads. You define an experiment template with actions (the faults), targets (the resources), stop conditions (CloudWatch alarms that abort the run), and an IAM role, then run it to observe how your application responds to failures such as instance termination, database failover, network disruption, or resource stress.
What is the difference between AWS FIS and AWS Resilience Hub?
Resilience Hub analyzes an application’s architecture against a resiliency policy and estimates whether it can meet its RTO and RPO targets, then recommends architecture changes, alarms, SOPs, and tests. FIS injects real faults so you can measure how the application actually behaves. Resilience Hub estimates; FIS measures. They are complementary, and Resilience Hub recommends FIS experiments as its tests.
How do stop conditions work in AWS FIS?
A stop condition references a CloudWatch alarm. If that alarm enters the ALARM state during the experiment, FIS stops the experiment and reverts any reversible faults. If the alarm is already in ALARM when you try to start, the experiment will not run. Stop conditions are the primary guardrail for limiting customer impact.
How do I limit the blast radius of an FIS experiment?
Select targets by resource tags and filters rather than broad matches, use the COUNT(n) or PERCENT(n) selection modes instead of ALL, scope the experiment IAM role to only the required actions and resources, set short action durations, and attach a stop condition on a steady-state alarm. Start in a non-production environment before running in production.
What is a resiliency policy in AWS Resilience Hub?
A resiliency policy defines target RTO and RPO values for each disruption type — application, infrastructure, Availability Zone, and optionally Region. When you run an assessment, Resilience Hub compares the application’s estimated RTO and RPO against the policy and reports whether the policy is met or breached.
Can AWS FIS simulate an Availability Zone outage?
Yes. The FIS scenario library includes an AZ Availability: Power Interruption scenario that combines multiple actions — affecting compute, networking, and managed databases in one AZ — to approximate losing an Availability Zone. You copy the scenario into an experiment template, tag or select the target resources, and add your own stop conditions.
Do I need to know FIS experiment JSON for the DOP-C02 exam?
You do not need to write templates from memory, but you should recognize the components — actions, targets, selection modes, stop conditions, and the IAM role — and know which one solves a given requirement. Scenario questions typically test whether you pick the managed feature (for example a stop condition) over a custom-built equivalent.
Conclusion
Resilience that has never been tested is a hypothesis. AWS Resilience Hub turns that hypothesis into a measurable assessment: describe the application, attach a resiliency policy with RTO and RPO targets, and let it report whether the architecture can meet them — then keep it honest with scheduled assessments and drift detection. AWS Fault Injection Service supplies the evidence: experiment templates that inject real faults into tagged, bounded sets of resources, with CloudWatch alarm stop conditions as the brake. For the DOP-C02 exam, remember the division of labour — Resilience Hub estimates, FIS measures — know the four parts of an experiment template, and reach for the managed guardrail before the custom one. Build both into your pipeline and recovery stops being something you hope for and becomes something you ship.