Horizontal vs Vertical Scaling

In plain words

When one machine is not enough, you have two moves. Scale up (vertical): replace it with a bigger one — more cores, more memory, a faster disk. Scale out (horizontal): add more machines like it and spread the work across them. Scaling up is the easy move and it runs out. Scaling out never runs out, and it changes what kind of system you have.

The two moves

Press "lose a machine" in the widget in each mode. That button is the argument.

What makes scaling out possible

A request can go to any copy only if every copy is interchangeable. That takes three things, and the first is the one that usually hurts:

1. State lives somewhere shared

   Sessions in Redis, uploads in object storage, nothing in local memory a user would miss. See stateless vs stateful.

2. Something spreads the work

   A load balancer for requests, a queue for jobs, a partition scheme for data.

3. Copies can come and go

   Health checks remove a sick copy; a new one boots from an image with no manual steps. If adding a server takes a runbook, you are not scaling out yet.

apiVersion: apps/v1
kind: Deployment
metadata: { name: api }
spec:
  replicas: 3                      # scale out: change this number
  template:
    spec:
      containers:
        - name: api
          image: shop/api:1.42
          resources:
            requests: { cpu: "2", memory:  }   # scale up: change these

N−1: the number that decides how many

You do not size for the load. You size so that the load is still fine with one copy gone — because one will be gone, during every deploy and every hardware failure.

copies at 70% carry the load

3

what the remaining 2 carry when one dies

105%

copies at 52% carry the same load

4

what the remaining 3 carry — fine

70%

The playground's "losing one instance" line in the inspector is this arithmetic for every part. The fix for a red N−1 is one more copy, not bigger ones.

Where it goes wrong

• Scaling out the wrong tier. Ten API servers in front of one database just move the bottleneck. Find the part that is actually full — the playground's Bottlenecks tab exists for this.
• Work that does not divide. One customer's data on one shard, one lock everyone takes, one queue partition for one hot conversation. More copies do nothing for these; see sharding.
• Scaling up past the knee. The largest instances cost far more per core than mid-sized ones. Past a point, three mid-sized machines are cheaper and more resilient than one huge one.
• Scaling the database like the app. Databases scale up first (it is simpler and single-writer designs need it), then out by replication for reads and sharding for writes. Do not skip the first step.

Take this with you

• The one idea: up is simple and has a ceiling; out has no ceiling and needs stateless copies, a balancer, and N−1 planning.
• In an interview, say which move for which tier, and give the N−1 arithmetic.
• At work, check what utilisation each tier lands at with one copy gone. If the answer is over 100%, you have one more copy to add.