# What if something breaks?

> Take a shop's database down, then its whole cloud region, see who notices, and try the fix in a proposal before you change anything.

Every system has a part it can’t do without. Most teams find out which one during an outage. This walkthrough finds it on purpose: you’ll pull the plug on a database, then on a whole cloud region, see how many people notice, and test the fix without touching the map.

You’ll use the **Checkout system** template, as in [Will it survive Black Friday?](https://docs.flozu.app/walkthroughs/black-friday.md). Open it from **Templates** with **Use template**, or carry on with the copy you made there.

> **Note**
>
> Flozu never touches your real systems. A drill is worked out from the lines and parts on the map, by fixed rules, so it plays the same way every time.

## 1. Take the database down

You break things inside a scenario, so start one first.

1. Open **Scenarios** and pick **Normal day**. Drag the playhead to about 20:00, the busiest part of the day.

2. Right-click **Orders DB** on the canvas and choose **Take down here**. It goes down at the playhead, for ten minutes.

![Right-clicking Orders DB on the canvas opens a menu with Take down here.](https://docs.flozu.app/_astro/take-down-light.DlzO3A8y_ZDxO17.webp)

## 2. See who notices

The Orders DB fades behind a dashed red outline. The line into it turns red, and its dots fade out before they arrive. Shoppers and the web shop show **33% fail** on their strips.

![The checkout map with Orders DB down: its card faded behind a dashed red outline marked Down, the SQL line into it red, Shoppers and the web shop showing 33% fail, and the panel saying up to 33% of Shoppers' traffic fails for 10 minutes.](https://docs.flozu.app/_astro/orders-db-down-light.qCEqZ1eO_1xFWCP.webp)

Why a third? Shoppers make 120 requests a second. 40 of those place an order, and every order is saved in the Orders DB. The other 80 are people browsing, which the web shop handles on its own, so they carry on as if nothing happened.

The panel’s summary says it in one sentence: up to 33% of Shoppers’ traffic fails, for 10 minutes. Open **Blast radius** to see what the failure touched and how much failed in all: about 24,000 requests.

> **Tip**
>
> A failure travels back along request lines, because the caller is waiting for an answer. Along a message line, only a send that fails on the spot counts, such as to a topic that’s down. Work that gets stuck further on never reaches the shopper: take the Email worker down for ten minutes, and Order events holds the emails until it’s back, without failing a single order.

## 3. Try the fix in a proposal

**Fix it** now starts with how to survive this. Nothing takes over when the Orders DB goes down, so it’s a **single point of failure**, and the fix is a change to the map’s shape: a standby copy of the database beside it.

1. Select **Try it** beside the fix. Flozu starts a [proposal](https://docs.flozu.app/guides/versions-and-proposals.md): a copy of the map with the standby database added, opened at the same moment of the same scenario.

2. Look at the top left. It says **Copes now**: with the standby, nobody notices the database going down.

3. The map itself hasn’t changed. Choose **Adopt** at the end of the island to make the proposal the map, or leave it for now.

## 4. Take a whole region down

A single database is one thing. What if everything in AWS eu-west-1 goes dark?

1. Go back to the map, open **Scenarios** and pick **Region down**. It takes the map’s busiest region down for half an hour at the day’s peak.

2. Everything in the region is dashed in red and marked **Down**. The summary says: when AWS eu-west-1 goes down, up to 33% of Shoppers’ traffic fails, for 30 minutes. That’s about 72,000 requests.

![Region down on the checkout map: the AWS eu-west-1 region and everything in it are dashed in red and marked Down, Shoppers and the web shop show 33% fail, the panel says up to 33% of Shoppers' traffic fails for 30 minutes, and the timeline marks the half hour it's down.](https://docs.flozu.app/_astro/failure-drill-light.Begv5NG0_eIzt8.webp)

The fix this time is a **second zone**: a copy of the region, with a copy of everything in it, standing by. Select **Try it** again, and the proposal says **Copes now**.

![A proposal with a standby copy of AWS eu-west-1 below the region that's down: the copy carries the traffic, the panel says nobody notices, and the top left says Copes now beside the proposal's name.](https://docs.flozu.app/_astro/proposal-verdict-light.Cly_0Ch9_VntVv.webp)

## 5. Find the weak spots you didn’t think of

You picked the database because it seemed risky. Flozu can check every part instead.

1. In any scenario, open **Weak spots** under **What happens** and select **Find weak spots**.

2. Flozu takes each part and zone down in turn, for an hour at the busiest moment, and lists them by how much fails.

![Weak spots on the checkout map: losing the web shop fails all of Shoppers' traffic, and AWS eu-west-1, Orders API and Orders DB each fail up to 33% of it, each with Drill this.](https://docs.flozu.app/_astro/weak-spots-light.GK3l50o8_ZmEqUR.webp)

The worst one isn’t in the cloud at all. Losing the **web shop** fails every request, because everything goes through it. The region, the Orders API and the Orders DB come next at a third each, then Payments and Order events at a tenth. **Drill this** adds that failure to the scenario so you can watch it play.

## What you learned

- **Who notices depends on the lines.** Requests carry a failure back to the people waiting; messages only carry back a send that fails on the spot.
- **A single point of failure needs a change of shape,** not more capacity: a standby copy, a second zone, or a queue in front.
- **Proposals let you prove a fix first.** The map only changes when you adopt.

## Next

- [Make the case for a fix](https://docs.flozu.app/walkthroughs/make-the-case.md) turns this proposal into something to show your team.
- [Failure drills](https://docs.flozu.app/guides/failure-drills.md) covers degraded parts, cut lines and every kind of fix.