← Concepts

Measuring systems

Foundations

APIs and Communication Styles

How parts of a system talk: request/response over HTTP, streaming over a held connection, or a message dropped on a queue — and when each is the wrong choice.

Polling is simple and wastes requests; pushing is efficient and means holding connections. Synchronous calls are easy to reason about and chain failures; asynchronous ones decouple and complicate.

RESTgRPCWebSocketpollingasync

Try it

Move the dials — the sentence under the picture changes.
Style
Requests or messages per second20.0 K/s · 333/s carry an updateTypical staleness2.5 sConnections held open0
100.0 K clients asking every 5 s is 20.0 K requests/s, of which only 2% carry news. Updates arrive up to 5 s late. Halve the interval and you double the load for a little less staleness.

In plain words

Every arrow in a system diagram is a conversation, and there are only a few ways to have one. Request/response: ask, wait, get an answer. Push: keep a line open and the other side speaks when it has something. Message: leave a note in a queue and walk away. Each is right for some arrows and expensive for others, and most bad architectures are one style used for everything.

Request/response: the default

The client asks, the server answers, the connection is done. HTTP is this, and so is almost every API. Two main dialects:

REST over HTTP + JSON
the default

Resources as URLs, verbs as methods, JSON bodies. Readable, debuggable with curl, cacheable by every proxy on earth. Slightly verbose, no schema unless you add one (OpenAPI).

gRPC
service to service

A schema (protobuf), generated clients, binary encoding, streaming built in. Faster and stricter; not readable, awkward from a browser. The usual choice between your own services.

GraphQL
many clients, many shapes

The client says which fields it wants; one round trip instead of five. Great for a product with many screens; a resolver that hides ten database calls behind one query is a latency trap.

The same operation, RESTHTTP
POST /orders HTTP/1.1
Content-Type: application/json
Idempotency-Key: …

{"sku": "boots-10", "qty": 1}

HTTP/1.1 201 Created
Location: /orders/8812
…and gRPC: a schema, and generated code on both sides
service Orders {
  rpc Create (CreateOrderRequest) returns (Order);
  rpc Watch  (WatchRequest)       returns (stream OrderEvent);   // server streaming
}

Getting updates: poll, long-poll, push

When the server has news the client did not ask for, request/response strains. Three ways out, and the widget shows the trade between them:

Polling
simple, wasteful

Ask every T seconds. Trivial to build, works through any proxy. Requests scale with clients ÷ T whether or not anything changed; updates arrive up to T seconds late.

Long polling
a bridge

Ask, and the server holds the request open until it has news (or 30 s pass). Near-instant updates over plain HTTP; the server now holds a request per client.

Push (WebSocket, SSE)
real-time

One held connection, messages either way. One frame per event — nothing wasted. The cost moves from request count to open connections, which is a different resource to size. See the chat problem.

BrowserServerGET /statspolling{count: 41}GET /stats (2 s later){count: 41} — nothing changedGET /stats (2 s later){count: 41} — still nothing── switch to WebSocket ──upgrade: websocket (held open){count: 42} — only when it changes
Polling asks and asks; push waits and speaks once. Same information, very different load.

Server-sent events (SSE) is the middle ground people forget: one-way push from server to browser over ordinary HTTP, with automatic reconnect. For "the server tells me things" with nothing to send back, it is simpler than WebSockets.

Synchronous or asynchronous

The other axis: does the caller wait?

Synchronous call

Checkout calls payments, waits for the answer, then calls email, waits, then calls the warehouse, waits. Simple to reason about; the response is complete. But checkout is as slow as the sum, as available as the product (see nines), and a slow email provider is a slow checkout.

Asynchronous message

Checkout writes the order, publishes "order placed" to a queue, and answers in 50 ms. Email, warehouse and analytics consume it in their own time. Decoupled and fast — and now "the order is placed" and "the email is sent" are different moments, and the client has to live with that.

Where it goes wrong

  • Chatty APIs. A screen that makes 40 calls to render pays 40 round trips. Batch endpoints, or GraphQL, or a backend-for-frontend that aggregates.
  • Polling at scale. Fine for 100 clients; at 100,000 it is a denial of service you run against yourself. The widget's first bar.
  • Push without reconnect logic. Every held connection drops on a deploy. Without jitter, they all come back at once — see retries and backoff.
  • Sync calls in a loop. "For each item, call the pricing service" is N round trips where one batch call would do.
  • Async without idempotency. A message can arrive twice. Every consumer needs the idempotency sentence.

Take this with you

  • The one idea: each arrow is a choice — ask/answer, push, or message — and the choice decides latency, load and what fails together.
  • In an interview, label the arrows and say which calls are synchronous because the caller needs the answer, and which are messages because it does not.
  • At work, count the polling loops. Each one is load that grows with users and carries almost no information.