Caching: fast answers, and the price of keeping them fresh
A cache keeps a copy of data somewhere faster to reach than its source. Done well, it turns a 20 ms database query into a sub-millisecond lookup and lets one database serve ten times the traffic. Done carelessly, it serves stale data, falls over all at once, or melts on one popular key. This module covers where to cache, the standard patterns, eviction, invalidation and the failure modes interviewers love to ask about.
1. The idea
Caching works because access is skewed: a small fraction of the data gets most of the requests (today's news, a celebrity's profile, the home page). Keep that fraction in memory and most requests never reach the slow source.
- Hit: the data is in the cache. Miss: it isn't, so you fetch from the source (and usually store it).
- Hit ratio = hits ÷ all requests. It decides everything: at 90%, the database sees 10% of reads; at 99%, 1%.
- TTL (time to live): how long an entry may be served before it expires.
- Eviction: what to throw out when the cache is full.
- Invalidation: removing or updating a cached copy when the source changes. The hard part.
Where caches live
| Layer | What it is | Good for |
|---|---|---|
| Client | Browser HTTP cache, app storage | Static files, rarely changing data; zero network. Little control once sent. |
| CDN | Edge servers near users (Module 10) | Images, video, scripts, public pages and API responses shared by many users |
| In-process | A map inside each app server | Tiny, very hot data: config, feature flags, the hottest keys. Fastest, but each server has its own copy. |
| Distributed cache | Redis, Valkey, Memcached, shared by all servers | The default application cache: sessions, query results, computed objects |
| Database | The database's own buffer cache, materialised views | Happens anyway; size memory so the working set fits |
2. How it works
Cache-aside, and its classic race
Cache-aside (also called lazy loading) is the default: the application checks the cache, and on a miss reads the database and fills the cache. On a write, it updates the database and deletes the cached key. Press Play, including the race at the end.
The other patterns
Cache-aside isn't the only way to wire a cache to a database. The four alternatives differ in two questions: who loads the cache on a miss (the application or the cache itself), and when a write reaches the database (before the caller gets an answer, or later). Pick a pattern below; each numbered arrow matches a numbered line, and pointing at a line lights up its arrow.
| Pattern | How | Use when | Watch out |
|---|---|---|---|
| Cache-aside | App reads cache → on miss reads DB, fills cache; on write updates DB, deletes key | Almost always; the cache holds only what's read | First read after a change is slow; delete-vs-fill races |
| Read-through | The cache itself loads from the DB on a miss | CDNs; caching libraries with a loader function | Needs a cache that knows your data source |
| Write-through | Every write goes to the cache and the DB together | Reads must see writes immediately | Slower writes; caches data that may never be read |
| Write-behind | Write to the cache; flush to the DB later in batches | Very high write rates where losing a little is acceptable (counters, analytics) | A cache crash loses unflushed writes |
| Write-around | Writes skip the cache; reads fill it | Data written once and rarely read soon (logs, uploads) | Recent writes always miss |
Eviction
- LRU (least recently used): evict what hasn't been touched for longest. The default; good for most workloads.
- LFU (least frequently used): evict what's used least often. Better when popularity is stable, and resists a one-off scan flushing the cache.
- TTL isn't eviction, it's freshness: entries expire after a time. Combine it with an eviction policy, or a full cache simply refuses writes (in Redis and Valkey, set
maxmemoryand a policy such asallkeys-lruorallkeys-lfu).
When caches go wrong
| Problem | What happens | Fixes |
|---|---|---|
| Stampede (thundering herd) | A hot key expires and thousands of requests miss at once and all hit the database | Request coalescing (one request rebuilds, the rest wait for it); refresh hot keys early, before expiry, or probabilistically; serve the stale value while one request refreshes |
| Hot key | One key (a viral post) gets so much traffic it overloads the single cache node that holds it | An in-process cache of that key on each app server; copies under several keys (post:42#1…#8) read at random |
| Stale data | The source changed, the cache didn't | Delete on write; short TTL as a safety net; versioned keys (user:7:v12) bumped in the same transaction |
| Cold start / cache down | An empty or failed cache sends 100% of reads to a database sized for 5% | Warm the cache before taking traffic; a circuit breaker and rate limit in front of the DB; replicas of the cache |
| Penetration | Requests for keys that don't exist miss every time | Cache "not found" briefly; a Bloom filter of existing keys |
Cache invalidation: keeping the copy honest
A cache holds a copy, and a copy goes out of date the moment the original changes. Invalidation is how you decide that a cached entry can no longer be trusted. There are three ways to do it, and real systems usually combine them.
| Approach | How it works | How stale can a read be? | Cost |
|---|---|---|---|
| Expire on a timer (TTL) | Every entry carries a time-to-live; after it, the next read misses and reloads | Up to the whole TTL | None to build; you trade freshness against hit ratio when picking the TTL |
| Delete on write | The code that writes the database also deletes the cached key | Usually milliseconds; longer if the delete fails or races a slow reader | Every write path must remember to do it, including scripts and admin tools |
| Invalidate from the change log | A separate worker reads the database's change stream and deletes the affected keys | The lag of the change stream, typically under a second | One more moving part to run; but no write path can forget |
The third approach is the most robust, because it watches the database itself rather than trusting every piece of code that writes to it. Here it is for a bookshop changing a book's price:
Whatever you choose, keep a TTL on every key as a safety net: if a delete is lost, the stale entry still dies on its own. And prefer deleting a key to overwriting it on a write; two writers overwriting in the wrong order leave the older value cached, while two deletes can't conflict.
Many caches, one truth
Large services often keep a small cache inside each application server (a few seconds of the hottest keys, read in microseconds) in front of the shared cache. That creates a new problem: when one server changes a value, the copies held by all the other servers are now wrong, and the database doesn't know they exist. Keeping these copies in agreement is called cache coherence.
- Short local TTLs (1–5 seconds) are the simplest fix: a local copy can only be wrong for that long. Good enough for prices on a listing page; not for a bank balance.
- Broadcast invalidations over a pub/sub channel, as above. Messages can be lost (pub/sub usually doesn't store them), so keep the short TTL as well.
- Versioned keys: store a version number in the shared cache and include it in the local key (
book:42:v7). A server that sees a newer version knows its copy is old. - Don't cache locally what must always be correct. Many teams only cache data locally that is read far more than it changes and where a few seconds of staleness is acceptable.
3. Try it yourself
How big must the cache be?
Popularity in real systems often follows a Zipf curve: the 2nd most popular item gets half the traffic of the 1st, the 3rd a third, and so on. See how much traffic a small cache absorbs, and what's left for the database.
LRU vs LFU, request by request
A cache with 3 slots. Type a sequence of keys (or use a preset) and compare the policies. The "scan" preset shows a one-off sweep through many keys flushing LRU's popular items.
4. Trade-offs
Speed vs freshness
Longer TTLs mean more hits and staler data. Set the TTL from the requirement: "prices may be 30 seconds old" means a 30-second TTL, plus active invalidation for anything that must be exact.
A new thing that can fail
The cache needs capacity planning, monitoring and a plan for when it's empty or down. The database must survive a cold cache, or you need to protect it.
Don't cache what must be exact
Live seat availability, account balances and inventory at checkout should be read from the source of truth. Cache the event page, not "is seat 14C free?".
Sometimes the index is the cache
An indexed lookup on a well-sized database takes a millisecond or two. Add a cache for expensive queries and real read pressure, not by reflex.
5. In real systems
- Redis is the most common distributed cache, with rich data types (hashes, sorted sets, streams). Since Redis 8 it's available under a choice of three licences, including the open-source AGPLv3 (redis.io licences).
- Valkey is the fully open-source (BSD) fork of Redis run by the Linux Foundation, compatible with Redis clients; version 9 was released in September 2026 (valkey.io). Many cloud providers' managed caches offer it.
- Memcached is a simpler, multi-threaded key-value cache: strings only, no persistence, very fast.
- Managed services (Amazon ElastiCache, Google Memorystore, Azure Managed Redis) handle replication and failover for you.
- Rough capacity: one node handles 100k+ simple operations a second at sub-millisecond latency; a cluster spreads keys across many nodes (Module 15).
- Large-scale practice: Facebook's Scaling Memcache paper describes leases to prevent stampedes and stale sets, and invalidation driven by the database's change log, the same ideas as this page at billions of requests per second.
The technologies up close
profile reads are 100× writes and hit the same few profiles), then in five steps:
- Bottleneck: read-heavy load, an expensive query, a tight latency target.
- What to cache: read often, changes rarely, costly to compute; name the keys (
user:123:profile). - Pattern:
Redis, cache-aside
, plus the CDN for media and in-process for very hot keys. - Eviction and freshness: LRU with a TTL set from the staleness requirement, delete on write.
- One or two failure modes that fit this system: stampede on hot keys, staleness for correctness, the cache going down.
Redis (and Valkey)
The default cache · know it wellAn in-memory data-structure server: commands run one at a time on a single thread (so no locks), about 100,000+ operations a second per node at sub-millisecond latency. The most versatile tool to know well, and not a system of record.
How it works
- Data structures: strings, hashes, lists, sets, sorted sets, streams, geospatial indexes, plus Bloom filters, JSON and time series in Redis 8.
- Durability gaps: RDB snapshots lose everything since the last snapshot; the append-only file (AOF) fsyncs every second by default, so up to about a second can be lost. Replication is asynchronous, so a failover can lose acknowledged writes. Need a durable Redis? AWS MemoryDB.
- Cluster: keys hash to 16,384 slots spread over nodes. Clients cache the slot map; a node that doesn't own a key answers
MOVED. There's no query router, so a multi-key command must touch keys in one slot; hash tags ({user:123}:posts,{user:123}:likes) force related keys together. - Batching: pipelining and
MGETmake many small lookups cheap.
| Use | How | The catch |
|---|---|---|
| Cache | product:123 → JSON or a hash, TTL per key | TTL handles freshness, not memory: set maxmemory and an eviction policy, or writes fail when full |
| Leaderboard | Sorted set: ZADD a score; ZREVRANGE for the top N | Trim unbounded sets (ZREMRANGEBYRANK) |
| Rate limiting | INCR + EXPIRE in one Lua script (fixed window), or a sorted set of timestamps (sliding window) | Setting EXPIRE on every call means the window never resets |
| Nearby search | GEOADD, GEOSEARCH … BYRADIUS (Module 12) | Post-filter by exact distance |
| Work queue | Streams: XADD, consumer groups, acks; reclaim a dead worker's items | Items can run twice: make processing idempotent. Long retention and replay → Kafka |
| Pub/sub | PUBLISH/SUBSCRIBE; sharded pub/sub in Cluster | At-most-once: offline subscribers miss messages (Module 19) |
| Distributed lock | SET lock:seat:14C <token> NX EX 30; release only if the token matches | An efficiency tool, not a correctness guarantee: failover or a paused holder can let two clients in (Module 21) |
Status today (checked October 2026)
Redis 8 is available under three licences, including the open-source AGPLv3; Valkey, the Linux Foundation's BSD-licensed fork, is client-compatible and reached version 9 in September 2026. Managed: ElastiCache, Memorystore, Azure Managed Redis, plus MemoryDB for durability.
Use it when
- Caching hot reads in front of a database
- Counters, rate limits, leaderboards, sessions, geo lookups
- Light queues and fan-out when Redis is already there
Skip it when
- The only copy of important data (it's not a system of record)
- Working sets that don't fit in memory affordably
- Joins or queries across keys on different nodes
- Durable, replayable streams for many consumers (Kafka)
In the interview
Redis, cache-aside, LRU with a 10-minute TTL, delete on write.
For a leaderboard: a Redis sorted set per board,
ZADD on each score, top 10 with ZREVRANGE.
- Redis as the database of record for payments or orders.
- A Redis lock guarding correctness (no double-booking) without fencing tokens or a database check. Use a row lock or conditional write for correctness.
- Multi-key operations across a cluster without hash tags: they fail with a CROSSSLOT error.
- No eviction policy: TTLs alone don't stop memory filling up.
Memcached
Situational · plain, huge cachesA simple, multi-threaded key-value cache: strings only, no persistence, no replication, very fast.
How it works
Clients spread keys over servers with consistent hashing; a lost server just means misses. Facebook's "Scaling Memcache" paper describes running it at billions of requests a second, with leases against stampedes and invalidations from the database's change log.
Status today
Mature and still used at very large scale; managed in ElastiCache and Memorystore. Redis has largely replaced it for new designs because of its data structures.
Use it when
- Very large, simple key → blob caches where multi-threaded throughput per node matters
Skip it when
- Anything needing data structures, persistence or replication
In the interview
Fine to say Redis or Memcached
; pick Redis if you need sorted sets, counters or streams.
- Choosing Memcached, then needing leaderboards or atomic counters it doesn't provide.
In-process (local) caches
Situational · an add-on for the hottest keysA small cache in each app server's own memory (Caffeine, Guava, an LRU map). The fastest possible read, but not shared.
How it works
Each server keeps its own copy with a short TTL. A hot key served locally never reaches Redis, so one celebrity profile can't overload one cache node. The price: each server may hold a slightly different version for a few seconds.
Status today
Common in every language; often a two-level setup: in-process (seconds) in front of Redis (minutes).
Use it when
- Small, hot, rarely changing data: configuration, feature flags
- Hot keys that would overload one Redis node
- A fallback when the shared cache is down
Skip it when
- Data that must be identical on every server immediately
- Large datasets (memory per server × number of servers)
In the interview
For the few hottest keys, a 5-second in-process cache on each server, so they never hit a single Redis node.
- In-process caches as the main cache: every server warms separately and invalidation becomes per-server.
Browser and CDN caching
The default for static and shared contentResponses reused by the user's browser or by CDN edge servers near users, controlled with HTTP headers.
How it works
Cache-Control says who may store a response and for how long; ETag lets clients revalidate cheaply (304 Not Modified). A CDN turns a 250 ms cross-world fetch into roughly 20–40 ms, and absorbs most traffic for popular objects (Module 10).
Status today
Universal: Cloudflare, Fastly, Akamai, CloudFront and the clouds' own CDNs.
Use it when
- Images, video, scripts
- Public responses many users share (catalogue pages, public posts)
Skip it when
- Personal or permission-checked data at the CDN (unless the CDN validates signed URLs)
- Live values like seat availability
In the interview
Media and public pages are served through the CDN with long TTLs; per-user data goes through Redis.
- Caching personal data at the CDN under a shared key: users see each other's data.
Staleness as a budget, per kind of data
The default · every cached value"Is a stale cache OK?" has no single answer. Each kind of data has its own budget: how old a value may be before showing it is wrong. Write the budget down as a number, and the TTL, the invalidation method and where you cache all follow from it.
How it works
| Data (an online bookshop) | Stale budget | What a stale value costs | So we… |
|---|---|---|---|
| Book descriptions, cover images | Hours | A typo fix shows late | Cache at the CDN with long TTLs; versioned image URLs |
| Bestseller list | 10 minutes | A slightly old ranking; nobody can tell | Recompute on a timer; serve from cache only |
| Price on the product page | About a minute | A shopper sees £9 and pays £7 (or the reverse: a complaint) | Short TTL plus delete-on-change; checkout re-reads the price from the database |
| Stock count shown | Seconds, shown as "few left" | "In stock" on an item that just sold out | Cache a rounded band, not the number; the order step checks the real count |
| Your own basket and orders | Zero for you, after you change it | "I added it and it vanished": a lost sale and a support ticket | Read your own recent writes from the primary, or from the client's local state |
| Account balance, permissions | Zero | Money or access granted wrongly | Don't cache, or cache only with a version that the write checks |
Three patterns keep staleness safe:
- Cache for display, check at the decision. Pages may show a cached price or stock level; the step that takes money or commits stock reads the source of truth.
- Cache where the requirement changes. A whole product page tolerates minutes; the price inside it doesn't. Cache the page shell long and fetch the price separately, rather than giving the whole page the strictest budget.
- Make the age visible and bounded. Every cached entry has a TTL even when it is also invalidated on change, so a lost invalidation heals within the budget. Where it helps users, show it: "prices as of 10:42".
Status today
This is how large shops and booking sites work in practice: browsing paths tolerate seconds or minutes, and payment, stock reservation and permissions always go to the source of truth.
Use it when
- Choosing TTLs
- Deciding which data the CDN may hold
- Answering "what if the cache is stale?"
Skip it when
- Nothing: every cached value has a budget, even if it's "zero, so don't cache"
In the interview
Prices can be a minute stale on the product page, so a 60-second TTL plus delete-on-change; but checkout always reads the price from the database, so a stale cache never charges the wrong amount.
- One TTL for everything.
- Charging, reserving or authorising from a cached value.
- Invalidation with no TTL behind it, so a single lost delete is stale forever.
6. Remember it
How this hook works: what the counter and cupboard mean
What the slogan means, phrase by phrase
- "Check the counter"
- Look in the cache first: fast, small, holds the popular items.
- "then the cupboard"
- On a miss, read the source of truth and put a copy on the counter (cache-aside).
- "clear the counter when you restock"
- On a write, update the database, then delete the cache key. The next read refills it with the fresh value. Deleting is safer than writing the new value into the cache, which can race with other writers.
Why this picture fits: a kitchen counter has the same constraints as a cache: limited space (eviction), copies of things kept elsewhere (the source of truth), and the risk of using an old jar after the cupboard was restocked (staleness).
The story → the concept
| In the kitchen | In the system |
|---|---|
| The counter | The cache (Redis, in-process) |
| The cupboard | The database, the source of truth |
| Putting a jar back to make room | Eviction (LRU, LFU) |
| A best-before date on the jar | TTL |
| Removing the old jar after restocking | Invalidation on write |
| Ten cooks rushing to the garage at once | A cache stampede |
Watch it on an example
- A product page reads
product:42from Redis: hit, 0.3 ms. - The price changes: the service updates Postgres, then
DEL product:42. - The next read misses, reads the new price from Postgres, and stores it with a 10-minute TTL.
- If the delete was lost (a crash between the two steps), the TTL caps how long the old price can be shown: the safety net.
The trap it saves you from
Updating cache and database as two independent writes. If they interleave with another writer, the cache can end up holding the older value forever.
It still works when…
The "counter" is a CDN: check the edge, fall back to the origin, and purge (or version the URL) when you publish a new file (Module 10).
7. Common mistakes
8. Design exercise (25 minutes)
- Pick what to cache. For a social app: user profiles, the home feed, like counts, "is this user following that user?", direct messages. For each: cache or not, where, TTL, and how it's invalidated.
- Size it. 10M users, 1 KB per profile, 20% of profiles get 90% of reads. How much memory does the cache need? One node or several?
- Break it. A celebrity's profile gets 200,000 reads a second. Which problem is this, and which two fixes would you combine?
- Try Redis locally (optional).
docker run -p 6379:6379 valkey/valkey, then invalkey-cli:SET p:1 hello EX 10,TTL p:1,GET p:1after 10 seconds. - Write it down: the cache-aside read and write steps, and three failure modes with one fix each.
9. Self-check quiz
In cache-aside, what should a write do to the cache?
A popular key expires and the database suddenly gets thousands of identical queries. What is this?
A nightly report reads every product once. Which policy best keeps the truly popular products cached?
Which should not be served from a cache?
In Redis Cluster, MGET user:1 user:2 fails. Why, and how do you make related keys work together?
Is a Redis SET NX lock enough to guarantee a seat is never double-booked?
A product page shows a cached price (60-second TTL). How do you make sure nobody is charged a stale price?
10. Go deeper
| Resource | Time | Why |
|---|---|---|
| Scaling Memcache at Facebook (NSDI 2013) | 1 h | Stampedes, stale sets, leases and cross-region invalidation at enormous scale; still the classic. |
| Caching challenges and strategies (Amazon Builders' Library) | 25 min | How Amazon decides when to cache, and protects services from cache failures. |
| Key eviction (Redis docs) | 15 min | The maxmemory policies and how Redis approximates LRU and LFU. |
| Caching (Hello Interview) | 20 min | How to bring caching into an interview: what to say, and which problems to raise. |
11. Under the hood (optional)
How caches really decide what to keep, expire and refill, and where they go wrong. Not needed for most interviews; open it when you want to know how the products really do it. All the deep dives are listed on the Under the hood page.
Caching internalsHit-ratio maths, eviction policies with a simulator, expiry, stampedes, the stale-fill race, hot keys, HTTP caching, 6 products comparedabout 35 min
A cache is a bet that the recent past predicts the near future. How it decides what to keep, when entries expire, and what happens in the moment between a miss and a refill decides whether it helps or quietly serves wrong data.
Hit-ratio arithmetic
The hit ratio works on misses, and misses are what reach the database. Going from 90% to 99% looks small but cuts database load tenfold.
Where the cache sits in reads and writes
| Pattern | Read | Write | Watch out for |
|---|---|---|---|
| Cache-aside (lazy loading) | App checks the cache; on a miss reads the database and fills the cache | App writes the database, then deletes the cache entry | The fill race below; the default choice |
| Read-through | The cache library or service loads misses itself | As cache-aside | Same as cache-aside, logic in one place |
| Write-through | From the cache | Write goes to the cache and the database together | Every write pays both; cache fills with data nobody reads |
| Write-behind (write-back) | From the cache | Write to the cache; flush to the database later in batches | Data loss if the cache dies before flushing |
| Refresh-ahead | Popular entries are reloaded before they expire | Unchanged | Wasted reloads for entries that went cold |
Eviction: what to throw away
When the cache is full, the eviction policy picks a victim. Real workloads are skewed (a few keys get most requests) and occasionally interrupted by scans (a batch job or crawler touching thousands of keys once). Run each policy on the same trace:
| Policy | How | Where |
|---|---|---|
| LRU | Evict the least recently used: a hash map plus a linked list | The textbook default; Memcached uses a segmented LRU (hot, warm, cold) |
| Sampled (approximated) LRU / LFU | Pick a few random keys (Redis samples 5 by default) and evict the best candidate among them: no list to maintain | Redis allkeys-lru, allkeys-lfu and the volatile-* policies |
| LFU | Evict the least frequently used; needs decay or old favourites stay forever | Redis LFU keeps an 8-bit logarithmic counter per key that decays over time |
| CLOCK, 2Q, ARC | Cheap LRU approximations (CLOCK) or two lists separating once-seen from repeatedly-seen keys (2Q, ARC) to resist scans | Operating-system page caches, database buffer pools |
| W-TinyLFU | A small LRU window admits newcomers; to enter the main cache a candidate must be more frequent than the victim, judged by a compact count-min sketch that halves periodically | Caffeine (Java), Ristretto (Go) |
| S3-FIFO | Simple FIFO queues: a small one for new keys, a main one, and a ghost list; most one-hit wonders never reach the main queue | Newer caches (2023 research) |
Expiry
- Lazy expiry: an expired key is deleted when someone reads it. Cheap, but keys nobody reads would linger forever.
- Active expiry: Redis also runs a background cycle that samples keys with a TTL many times a second and deletes expired ones, repeating while a large share of the sample is expired.
- TTL jitter: 10,000 entries filled at the same moment with the same TTL all expire together. Add a random ±10% so refills spread out.
- TTL is for freshness, not memory: without a
maxmemoryand an eviction policy, a full Redis rejects writes (the default policy isnoeviction).
Stampedes and request coalescing
- Single-flight (request coalescing): only the first miss for a key goes to the database; others wait for its result. Inside one process this is a map of in-flight futures; across servers, a short lock in the cache (
SET lock:k NX EX 5). - Serve stale while refreshing: keep the old value past its TTL for a grace period and refresh in the background (HTTP
stale-while-revalidate, Varnish grace). - Refresh early, probabilistically: each reader refreshes with a probability that rises as expiry approaches, so one reader refreshes before everyone misses (the "XFetch" algorithm).
- Negative caching: cache "not found" briefly too, or repeated lookups of missing keys (often attacks) go straight to the database.
The stale-fill race
Cache-aside with "delete on write" still has a gap: a slow reader can put an old value back after the writer deleted it.
Fixes: leases (the cache hands the first misser a token; a delete invalidates it, so the late fill is refused, as Facebook's memcache does); versioned values (only set if newer); invalidate from the database's change stream (change data capture), with a second, delayed delete as a cheap safety net; and always a TTL so any stale entry eventually dies.
Hot keys, two tiers and memory
- Hot keys: one key on one cache node can exceed that node's network or CPU. Replicate it under several names (
k#1…k#8) and read a random copy, or put a tiny in-process cache (seconds) in front. - Two tiers: in-process cache (microseconds, per server) in front of Redis (sub-millisecond, shared). Invalidate both, for example by publishing invalidations on a channel.
- Memory overheads: every Redis key costs tens of bytes beyond its value, so millions of tiny keys add up; small hashes, lists and sets use compact encodings (listpacks, intsets) until they pass a size threshold. Memcached stores items in fixed slab classes, so a change in value sizes can leave memory stranded in the wrong class (slab rebalancing moves pages over).
- Snapshots fork: a Redis RDB snapshot forks the process; pages written during the snapshot are copied, so leave headroom (up to double memory under heavy writes).
HTTP caching semantics
| Header | Meaning |
|---|---|
Cache-Control: max-age=60 | Fresh for 60 seconds in any cache |
s-maxage=600 | Overrides max-age for shared caches (CDNs, proxies) |
private / public | Only the browser may store it / shared caches may too |
no-cache vs no-store | Store it but revalidate before every use / never store it at all |
immutable | It will never change: don't even revalidate (for fingerprinted file names) |
stale-while-revalidate=30, stale-if-error=86400 | Serve stale while refreshing in the background / when the origin is failing |
ETag + If-None-Match | Revalidation: the server answers 304 Not Modified with no body if unchanged |
Vary: Accept-Encoding | Cache separate copies per value of that request header (too many values fragment the cache) |
Surrogate-Key / cache tags | CDN-specific tags so one purge clears every page showing a product |
Product by product
| Product | Memory and threading | Eviction and persistence | The detail worth knowing |
|---|---|---|---|
| Redis / Valkey | One thread runs commands (I/O threads help with sockets); compact encodings per data type | Sampled LRU or LFU; lazy plus active expiry; RDB snapshots (fork) and an append-only file | One slow command (KEYS, a huge range) blocks everyone |
| Memcached | Multi-threaded; slab allocator with fixed size classes | Segmented LRU per slab class; no persistence; clients shard with consistent hashing | Slab classes can starve when value sizes shift |
| Caffeine, Ristretto | Inside the application process | W-TinyLFU / TinyLFU admission | Fastest; every instance has its own copy to invalidate |
| Varnish, NGINX proxy cache | HTTP reverse-proxy caches | TTL from headers; grace serves stale while refreshing | Coalesce identical misses into one origin request |
| CDNs | Edge locations plus a tiered or shield layer | Purge by URL, tag or everything; cold content evicted | A shield tier cuts origin load; tags make purges precise |
| Browsers | Memory and disk cache per origin | Cache-Control, ETag revalidation, service-worker caches | Fingerprinted file names can be cached for a year |
Say it in the interview
- "What happens when a hot key expires?" A stampede: thousands of misses hit the database at once. Coalesce them (single-flight or a short lock), serve stale while one request refreshes, and add TTL jitter.
- "Can cache-aside serve stale data forever?" Until the TTL, yes: a slow reader can refill an old value after the writer's delete. Leases or versioned sets close the race; a TTL bounds it.
- "Why not plain LRU?" A single scan flushes the hot set. Frequency-aware admission (W-TinyLFU) keeps the hot set and refuses one-hit wonders.
The hit ratio rises from 90% to 99%. What happens to the load on the database?
What does Cache-Control: no-cache mean?
no-store is the one that forbids storing.How does Redis implement LRU eviction?
Sources
- Redis: key eviction
- Redis: EXPIRE and how keys expire
- Memcached wiki
- Caffeine: efficiency (W-TinyLFU)
- TinyLFU paper (Einziger, Friedman, Manes)
- Scaling Memcache at Facebook (NSDI 2013)
- Optimal probabilistic cache stampede prevention (VLDB 2015)
- MDN: Cache-Control
- RFC 9111: HTTP caching
- Fastly: surrogate keys
- Varnish: grace mode