Caching

Keeping a copy closer to the reader. The interesting decision is not where reads go but where writes go.

Every write strategy trades durability against how often you touch the store.

cache-asidewrite-throughwrite-backhit rate

Try it

Move the dials — the sentence under the picture changes.

Write strategy

Reads80% reads

Cache-aside invalidates on write, so the next read for that key pays a miss. Simple and safe, and it is why the hit rate sits at 0% rather than higher — writes keep knocking entries out.

What it is

A cache is a smaller, faster copy of data that lives closer to whoever is reading it. Every cache is a bet that the same things get read repeatedly, and access patterns in real systems are skewed enough that the bet almost always pays.

The read side is the easy half. The decision that matters is what happens on a write.

The write strategies

Cache-aside (lazy loading) is the default. The application reads the cache; on a miss it reads the store and fills the cache. On a write it updates the store and invalidates the entry. Simple, resilient — a cache outage degrades to slow rather than wrong — and it means every write costs the next reader a miss.

Write-through writes the cache and the store together, synchronously (in the same step, before answering the caller). The cache is never stale and a crash loses nothing, but every write pays for both, and you end up caching data that is never read.

Write-back (write-behind) writes only the cache and flushes to the store later. It absorbs write bursts beautifully and produces the fewest store writes. It is also the only one of the three where a cache failure loses acknowledged data.

Eviction

A cache that never evicts is just a slower database. LRU (least recently used: drop whatever nobody has touched for longest) is the usual answer and it is good until one scan of cold data evicts everything valuable. LFU (least frequently used) resists that but clings to things that were popular yesterday. TTL (time-to-live: an expiry stamp on each entry) is not really an eviction policy — it is a statement about how stale you are willing to be, and it composes with the others.

Invalidation

The hard part — the reason people joke that cache invalidation is one of the two hard problems in computer science. Invalidation means noticing that a cached copy has gone stale and getting rid of it. Three broad options:

TTL only. Simple, and you accept bounded staleness. Usually right.
Explicit invalidation on write. Correct until two writers race, or until the invalidate is lost because it went to a different cache node.
Versioned keys. Never invalidate; change the key. Elegant, and it needs a version you can cheaply derive at read time.

Some data sidesteps this entirely. A URL shortener's code-to-URL mapping is immutable once written, which is why caching it is nearly free.

Where it goes wrong

Thundering herd. A popular key expires and a thousand requests all miss and all hit the store at once. Fix with a lock on the fill, or by refreshing early.
Cache penetration. Requests for keys that do not exist miss every time and go straight through. Cache the negative result.
Cold start. A restarted cache fleet sends full read traffic at the store.
Hot keys. Splitting the cache across machines (sharding) distributes keys evenly, not traffic. One viral key still lands on one shard.

Take this with you

The one idea: a cache is a bet on repetition. Reads are easy; decide on purpose what a write does to the copy.
In an interview, say cache-aside with a TTL unless there is a reason not to, and name the thundering herd on a hot key expiring.
At work, find your hit rate and your worst key. A 95% hit rate means the store still takes 5% of reads — and 100% of them on a cold restart.