Designing a cache for a read-heavy API

Decide who populates the cache and what happens on a write.

Model answer. For read-heavy data, put an in-memory cache such as Redis or Memcached between the application and the database.

Cache-aside (lazy loading) — the most common pattern

1. On a read, the app looks up product:{id} in the cache.
2. Hit: return the cached value — sub-millisecond, no database work.
3. Miss: read from the database, store the result in the cache with a TTL, then return it.

Only data that is actually requested gets cached, and if the cache goes down, the app can still fall back to the database.

Keeping data fresh

• TTL: every entry expires after, say, 5–10 minutes, which bounds staleness.
• Invalidate on write: when a product is updated, write to the database first, then delete the cache key so the next read reloads fresh data. Deleting is usually safer than updating the cached value, because two concurrent updates could otherwise leave the older value in the cache.
• Alternatives: write-through (write to cache and database together, so the cache is always populated) and write-back (write to the cache and flush to the database later — fast but risks data loss).

Eviction: memory is limited, so the cache evicts entries using a policy such as LRU or LFU.

Failure modes to guard against

• Cache stampede (thundering herd): a hot key expires and thousands of requests miss at once and hit the database. Mitigate with a per-key lock or request coalescing so only one request rebuilds the entry, and add random jitter to TTLs so keys do not expire together.
• Cache penetration: repeated requests for IDs that do not exist always miss. Cache a short-lived "not found" result or use a Bloom filter.
• Stale reads right after a write, if invalidation fails — keep TTLs as a safety net.

Other layers: static assets and images belong on a CDN; browsers can cache via Cache-Control headers.

Common follow-up: "What should you not cache aggressively?" Data where staleness is costly — stock counts during a flash sale or payment status. Use short TTLs or read from the source of truth.