A look at how Instagram works. Instagram serves billions of users a feed that’s the textbook example of the “fanout-on-write” pattern, with a hybrid escape hatch for celebrities. The architecture that’s survived a decade of feature growth is built on one decision: each user gets a pre-materialized timeline in Redis, updated whenever someone they follow posts.
How Instagram Works: The problem at scale
Instagram’s feed is sorted by ranking, not time. When you open the app, the system needs to return a personalized, ranked set of posts from the hundreds of accounts you follow, within 200ms, before you abandon the app. Naive implementations (query all followees’ recent posts, rank, return) die at the join: a user following 500 accounts creates a 500-way join against the post table. Instagram’s solution is the opposite of “query on read” : it’s pre-compute on write, materialize each user’s timeline ahead of time, and serve from a cache.
The core architecture
When you post, the API writes the post to the canonical store, then triggers a Fanout Worker. The worker walks your follower list and appends the post ID to each follower’s Redis sorted set (their “timeline”). When a follower opens the app, the read path is just ZREVRANGE user:N:timeline 0 50, a single Redis call returning the latest 50 post IDs. Ranking ML then re-orders those 50 based on predicted engagement (how likely you are to like, comment, share, save, or dwell). Media assets (photos, videos) are served from a CDN; only the post metadata flows through the timeline cache.
Why fanout-on-write beats fanout-on-read
Read latency is everything for a feed product. A 500ms cold-cache read means a user closes the app. By pre-materializing timelines, the read is O(log N) on Redis (one sorted set lookup), not O(M × log N) joining M followees’ posts. The trade-off is write amplification: posting to 500 followers becomes 500 Redis writes; posting to 100 million followers (Taylor Swift) becomes 100 million Redis writes, which is unsustainable. This is where the architecture had to evolve.
The celebrity exception
Instagram’s published engineering blog describes a hybrid: for users with more than ~50,000 followers, the system skips fanout-on-write entirely. Their posts go to the canonical store but NOT to followers’ Redis timelines. When a non-celebrity user reads their feed, the system merges their Redis timeline with a separate query against the few celebrities they follow who use the fanout-on-read path. This keeps the write amplification bounded (even Beyoncé’s post only writes once) at the cost of a few extra DB reads per user timeline query.
The threshold (~50k followers) is empirically tuned. Below it, fanout-on-write is a net win; above it, the fanout cost exceeds the read savings. The crossover point is a function of average read frequency vs average write frequency, and it shifts as user behavior changes.
The one non-obvious insight
Ranking happens on the read path, not the write path. The fanout worker just appends to timelines in posting-time order; the Ranking ML re-orders on every read. This means ranking models can be updated continuously without backfilling 100 million Redis timelines: change the model, every read across the planet uses the new model within the next page load. If ranking were baked into the write path, every model change would require rebuilding the cache.
This is why Instagram can ship ranking improvements multiple times a week: the cache is post-IDs only, ranking is decoupled, and the model is the single point of change.
Instagram is the canonical fanout-on-write timeline, with a hybrid escape for celebrities. The two non-obvious bets (write amplification capped at 50k followers, ranking decoupled from caching) are what kept the architecture scalable through a decade of feature growth.