Bad deploys that roll themselves back.

The ten-minute watch was a real cost — engineers stopped what they were doing to babysit the dashboards. It was also unreliable; subtle regressions snuck past tired eyes. Some deploys got rolled back at the next customer report instead of the next minute.

The deployment circuit breaker rolls a deploy back automatically if specified CloudWatch alarms breach during the deploy window. We wired up alarms on error rate, p99 latency, and synthetic-canary health. A breach during deploy triggers immediate rollback to the previous task definition.

Bad-deploy detection landed inside two minutes. Auto-rollbacks happened roughly three times per month — each one a preventable incident the team didn’t have to investigate. The human watch became optional; most deploys go in unwatched, with the breaker as the safety net.