How traffic is worked out

Flozu works out everything from the numbers on the map. It never runs, calls or measures your systems. This page explains the model, so you know what a scenario’s answer means.

A worked example

Here’s Black Friday on the Checkout system template, at ten past midnight, its busiest moment. Shoppers send 7.3 times their busiest on a normal day. Every part passes on 7.3 times what it usually does, so every line’s rate is multiplied by 7.3.

Every number is multiplied by the same 7.3 at ten past midnight. The part with the least room on a normal day runs out first.

Then each part’s load is compared with what it can take. The Orders API gets 292 a second and can take 400: 72% busy, past the 70% it should stay under, so it’s tight. The Email worker gets 88 a second and can take 20: 438% busy, so it’s over. To bring it back under 70%, it needs 88 ÷ 0.7, about 126 a second. That’s the fix Flozu suggests.

The rest of this page is the rules behind each step.

At one moment

Start from where traffic enters: a part no line flows into, such as people or an app. A scenario scales what those parts send, then traffic moves from each line’s source to its target.

• A part sends on traffic in proportion to what reaches it, as on a normal day. If twice as much arrives, it sends twice as much down each of its lines. Where several lines meet, each counts by its rate. A two-way line counts once, from source to target.
• Past a part’s limit, the excess is turned away, so the parts after it never see it.
• Past a part’s capacity, work backs up and its buffer fills. The traffic still flows on as if the part kept up, so one run sizes a whole path rather than only the first bottleneck.
• A part that auto-scales runs the fewest copies that keep each one under the spare room target. Its capacity is then its capacity per copy times the copies it runs.
• Loops settle after a few passes. If traffic keeps growing round one, the result says it didn’t settle.

Rates are compared as plain numbers, whatever their unit.

How busy a part is

Status	Means
OK	Within its spare room.
Tight	Past the spare room target, 70% busy by default.
Over	Past its capacity. Work backs up.
Down	A failure took it down, so everything that reaches it fails.
Not checked	No capacity set, so there’s nothing to compare against.

On the canvas, a part’s health strip splits these into finer zones, from underused to over.

A part is flagged when it’s tight, over or turning traffic away. Each flagged part comes with a fix: the capacity that brings it back under the spare room, how many times today’s that is, and, for a capacity per copy, how many copies to run. For a limit, it’s what to raise the limit to.

Over time

A scenario over time plays its timeframe in steps, from every 5 seconds up to 5,000 steps, short enough to catch its quickest event. At each step it works out the moment as above, and carries two things from one step to the next:

• Copies take time to start. A part that auto-scales asks for the copies it needs at once, but a new copy only takes traffic once it’s started. A spike faster than that runs the part over until its copies are up.
• Buffers fill and drain. A part with a buffer keeps what it can’t process waiting, up to its buffer, and turns the rest away. Once the rush passes, it works through what’s waiting at full capacity.

The day’s rhythm

A part where traffic starts is as busy as its share of the day. With a day’s rhythm, it runs from its quietest number of people to its busiest across the day, quietest at about 4am and busiest at about 8pm, shaped so the day averages out at its typical number. Without people set, its quietest is 15% of its busiest.

Every level in a scenario is a multiple of a normal day’s busiest moment.

When something fails

A scenario can take a part, a line or a whole zone down for a while, or leave it degraded with only part of its capacity. Flozu then works out where its traffic goes, by rules on the map’s own lines and kinds:

• A part that’s down takes nothing. Everything that reaches it fails, and it sends nothing on. A degraded part keeps part of its capacity, or of its copies if it auto-scales.
• A zone that’s down takes every part in it down, and every part in the zones inside it.
• A cut line carries nothing, and what it would have carried fails where it starts.
• Failures travel back on requests. A caller fails as much of its own traffic as its requests fail, back to where traffic starts, so you can say “18% of shoppers’ requests fail”. A message’s sender doesn’t wait, so it only notices a send that fails straight away.
• Traffic moves off what failed, where it can. A line that stands by, such as a failover to a replica, takes it over. A load balancer or DNS spreads it over the lines beside it that are still up, which can run them over. A dead cache’s reads fall through to the database behind it. A queue holds work for a part that’s down, until its buffer is full, then works through it when the part is back. Anything with nowhere to go fails.
• Coming back is cold. A part that auto-scales comes back with no copies running, waits for its least copies to start, and only then scales out.

Here’s the Orders DB going down on the checkout. The Orders API waits on it, so its orders fail, and so on back to the shoppers. Orders are 40 of the 120 requests shoppers make each second, so a third of their traffic fails. Order events and the Email worker are reached by messages, so they’re out of it.

Requests carry a failure back to whoever is waiting. Messages don't wait, so trouble further down stays where it is.

Give the Orders DB a standby copy, and the standby line takes its traffic over the moment it goes down. Nobody notices.

A standby carries nothing until the line beside it fails, then takes its traffic over.

Failures follow fixed rules, with nothing random about them, so a drill plays the same way every time.

Comparing two maps

To compare a proposal with where it began, Flozu plays the same scenario on both, leaving out anything it reaches that only one of them has, and asks which went better. A saved scenario the proposal changed still plays as one copy on both, the proposal’s unless you pick the other, so the difference comes from the maps alone. A map that nobody notices anything on beats one where something goes wrong. When both run into trouble, it compares, in this order: how much fails where traffic starts and for how long, how long parts run short of room, how much they turn away, and how busy they get at their worst. Two maps within 1% of each other are as good as each other.

Randomness that repeats

Random spikes and wobble come from a seed, so a scenario always plays the same way. That’s what lets you show a run again in a presentation or a share link. Roll again picks a new seed. The odds play many seeds and sum them up.

Estimates

From the load at each moment, Flozu also estimates how long requests take, how much fails and what the map costs. Response times and cost shows where they appear. The rules:

• Each part takes a time of its own to handle one thing: the time it’s given, or a typical time for its kind.

• It slows down as it gets busy, by the usual queueing rule: its time divided by how much room it has left, time ÷ (1 − how busy). At 95% busy or more, it stops at 20 times. Work waiting in its buffer adds the time to work through it. For a part that takes 50 ms:

  How busy	Takes	So
  0%	1 × its time	50 ms
  50%	2 ×	100 ms
  75%	4 ×	200 ms
  90%	10 ×	500 ms
  95% or more	20 ×	1 s

  That’s why the spare room matters: the busier a part gets, the more each extra bit of load slows it down. Going from 50% to 75% busy doubles its time; going from 75% to 95% multiplies it by five.

• A request times out after 30 seconds. A part over its capacity with no buffer falls further behind, so what it can’t get to times out, and the rest waits as long as it can.

• A request’s time adds up along the requests it makes. A part’s time includes the time of each part it calls, counted by how many calls it makes for each thing it handles, by the lines’ rates: three queries a request count three times, and a cache that sends a fifth of what it gets on to the database counts a fifth. A load balancer sends each request down one line only, so each line counts its share. Messages don’t wait for an answer, so they add no time.

• What fails is everything a failure breaks, what a limit or a full buffer turns away, and what times out, here or along the requests a part makes. A request’s time only counts the ones that get through.

• Across a scenario, each moment counts by the traffic that gets through then. 95% of requests are done within twice the typical time, or within the time at the slowest moments when one request in twenty or more comes then.

• Cost adds up what each part costs an hour: once for a part, or once for each copy it runs or is starting, for one that auto-scales. A month at that rate is 730 hours.

What it can’t tell you

The model is deliberately simple. Its timeout is the same 30 seconds everywhere, it doesn’t know about retries, cold caches or slow queries, and a failure moves traffic only along the lines you’ve drawn. It’s best at finding which part runs out of room first and roughly how much more it needs, and at explaining that to someone. Treat its numbers, its estimates most of all, as a starting point for a conversation, not a guarantee.