A look at how ChatGPT works. ChatGPT served 100 million weekly active users within two months of launch, the fastest consumer-software adoption in history. The architecture that scaled to that size wasn’t new technology; it was the careful application of patterns every ML inference system has used for years, plus one trick: the per-session KV-cache… Continue reading
How ChatGPT Works: KV-Cache, GPU Cluster, and the Router
