Chaos Engineering Cheat Sheet

Principles and tooling for deliberately injecting failure into systems to validate resilience before real incidents occur.

2 PagesAdvancedFeb 20, 2026

Principles of Chaos Engineering

Core tenets from the Principles of Chaos manifesto.

Steady state hypothesis- Define a measurable baseline of normal system behavior before experimenting
Vary real-world events- Inject failures that mirror real risks: server crashes, network latency, dependency outages
Run in production (carefully)- Staging rarely reflects production traffic and topology accurately
Minimize blast radius- Start small and scoped, with an automated abort mechanism
Automate experiments- Run continuously, not as one-off manual exercises

LitmusChaos Pod Delete Experiment

Kubernetes-native chaos experiment manifest.

yaml

apiVersion: litmuschaos.io/v1alpha1kind: ChaosEnginemetadata:  name: nginx-chaosspec:  appinfo:    appns: default    applabel: app=nginx    appkind: deployment  chaosServiceAccount: litmus-admin  experiments:    - name: pod-delete      spec:        components:          env:            - name: TOTAL_CHAOS_DURATION              value: '30'            - name: CHAOS_INTERVAL              value: '10'            - name: FORCE              value: 'false'

Toxiproxy Latency Injection

Simulate network latency between services using Toxiproxy's CLI.

bash

# Create a proxy in front of the real servicetoxiproxy-cli create -l localhost:26379 -u redis:6379 redis_proxy# Add 500ms latency with jitter to all traffictoxiproxy-cli toxic add -t latency -a latency=500 -a jitter=100 redis_proxy# Remove the toxic once the experiment is donetoxiproxy-cli toxic remove -n latency_downstream redis_proxy

Common Failure Modes to Test

Categories of faults typically injected in chaos experiments.

Resource exhaustion- CPU, memory, or disk pressure on a node or container
Network faults- Latency, packet loss, DNS failure, partitions between services
Dependency failure- Downstream API returns errors or times out
Instance termination- Kill a pod, VM, or availability zone to test failover

Pro Tip

Always pair a chaos experiment with an automated abort condition tied to a real SLO metric — if user impact exceeds a threshold, the experiment should halt itself immediately, not wait for a human to notice.

Was this cheat sheet helpful?

Incident Response & Postmortems Cheat Sheet

Feature Flags Cheat Sheet