Zero to SGLang: the Official Inference Course

SGLang crossed 36,000 stars as one of the fastest-growing LLM serving engines, and until now the only way to learn its internals was the source tree. Zero to SGLang, the official course from the SGLang team with the Datawhale learning community, changes that: a structured path from your first deployed server to a hand-built mini engine, then down into the real SGLang’s machinery.

The course lives at datawhalechina/zero-to-sglang and opened in late August 2026, so everything here reflects its state in late September: English editions of Parts 0 and I are published, and the build chapters carry visible status markers as they land. That transparency is a feature, not a caveat, and we will get to how to plan around it.

Part 0: a server running before theory

The opening move is the right one for motivation. Chapter 0.2 walks environment setup and has you serving Qwen3-0.6B under SGLang on GPU hardware you control, in the first sitting, which means every later abstraction attaches to something you have already run. Chapter 0.1 pairs it with something rarer in systems courses: a working-ethics chapter about owning your code, communicating like a human, profiling before optimizing, and participating in open source.

The pairing is deliberate. A course produced with a community organization, and aimed partly at people who will contribute back to SGLang, benefits from stating its culture early. For readers of our inference learning path, Part 0 is the quickest on-ramp in our whole inference coverage: one evening, one running server, zero prerequisites beyond access to a GPU, and a concrete artifact running on your machine before any abstraction arrives.

Part I: the concepts, no code attached

The foundations part is designed to be readable with no GPU and no coding, which makes it the strongest section for the general audience. Five chapters build the vocabulary that engine documentation assumes you already hold:

Chapter	What it covers
Introduction to LLM	What a language model is, the Transformer, autoregressive generation
Introduction to Inference	Training versus inference, prefill and decode, compute-bound versus memory-bound, the Roofline model
Introduction to GPU	GPU architecture and where inference bottlenecks actually come from
KV Cache	Deriving the cache from attention, its lifecycle, quantitative memory analysis
Introduction to Benchmark	TTFT, TPOT, ITL, goodput, percentiles and tail latency, designing and reading benchmarks

The benchmark chapter deserves a callout because it is the one most courses skip. Serving conversations are full of latency acronyms that few people can define precisely, and this chapter turns them into precise, measurable definitions, then teaches how to design a benchmark run and read its distribution rather than its average, which is the specific skill that separates serving engineers who measure from those who guess. The habit generalizes past serving, too, because every infrastructure discipline eventually meets a metric someone quotes without a percentile, and being the person who asks which tail, at what load, is a quiet form of technical leadership. The GPU chapter, still in revision at write time, is the bridge into the build part.

Part II: build a mini SGLang in ten chapters

The center of the course is a from-scratch construction of a small inference engine, in the same spirit as the build-your-own courses we have covered, but targeted specifically at SGLang’s design. The published opening chapter maps an inference engine’s architecture and module breakdown; the build sequence then runs from a hand-written forward pass and generation loop in your first 200 lines, through the KV cache reducing attention from quadratic to linear cost, HTTP serving with concurrent requests, continuous batching and the scheduler, paged KV management, and then the two chapters that make SGLang itself distinct: RadixAttention with prefix caching, and multi-process execution with tensor parallelism, closing with speculative decoding.

Reading that list against a production engine’s feature set is the fastest way to see the course’s ambition: by the end you have built small versions of nearly every subsystem a serving engine ships. The build chapters were largely under construction at write time, with per-chapter status markers in the syllabus, so the practical pattern is to follow the repository and work each chapter as its marker turns green, and to treat the architecture chapter, already published, as the map that makes every later arrival meaningful the day it lands.

Part III: inside the real SGLang

The third part drops the training wheels and examines production SGLang directly. The planned chapters compare the mainstream attention backends, FlashInfer, Triton, FlashAttention-3, and FlashMLA, alongside CUDA Graph usage; quantization and low-precision inference; and hierarchical caching for long contexts. For engineers who operate SGLang rather than build it, this is the part that turns configuration decisions back into engineering decisions, which is the same value our serving engines field guide argues for from the operator side.

A fourth part rounds out the roadmap, and the course’s own status table is the source of truth for what is live on any given day. That table, plus per-chapter English and Chinese editions, is the artifact to bookmark rather than any snapshot of progress, ours included. A snapshot ages in weeks here; the status table does not, because the maintainers refresh it with every chapter they publish. Treat Part III as the long-term payoff: the chapters naming specific backends and kernel techniques are the ones that age into reference material you return to long after the build chapters are finished.

One more note for teams: the course doubles as shared vocabulary. A team that reads Part I together stops arguing past each other about latency, because the metric definitions give every argument a shared ruler, and that alone justifies the reading time before any engine decision.

How to follow a course that publishes progressively

Learning from a course under construction takes a small method shift. Follow the repository rather than downloading a snapshot, and let the syllabus markers drive your schedule: a chapter marked complete is a commitment you can plan a session around, a chapter in revision is preview material worth skimming, and a chapter under construction is a roadmap item, not a broken link. The English and Chinese editions move at different paces per chapter, so check the language badge alongside the status marker.

Two habits make the format an advantage. Working each chapter as it lands keeps you at the course’s own cadence, which is roughly the cadence of a study cohort, and the exercises are small enough that a week between chapters does not evaporate your context. And reading a chapter in revision, then returning after its update, teaches a subtle skill the finished-course world cannot: watching technical writing converge, which is exactly how engineering documentation actually matures in the workplace.

Who this course fits, honestly

Three audiences, three different entry points. Engineers already running SGLang in production should read Parts I and III and treat Part II as optional: the concepts and the internals chapters directly improve operational decisions. Learners who want inference systems depth and enjoyed the idea of building an engine should plan the full path, accepting that the build chapters arrive over time. And readers comparing this with a mini-vLLM build course should choose by engine: the mechanics overlap heavily, continuous batching, paged cache, scheduling, and the differentiators are SGLang-specific, RadixAttention and prefix caching first among them.

The prerequisite profile is modest at the entrance and steep through Part II: comfort with Python, a GPU for the deployment and build chapters, and enough C++ and CUDA reading ability that kernel-adjacent chapters do not stall you. The Chinese edition is the canonical text with English translations following chapter by chapter, so English-only learners should check a chapter’s language badge before scheduling it.

How complete is the course right now?

Incomplete by design, and precise about it. The syllabus carries a status marker per chapter: completed, in revision, and under construction are all visible, so a learner never has to guess whether a link will dead-end. At write time, English Part 0 and most of Part I are complete, the GPU and benchmark chapters are in revision, the Part II architecture chapter is published while the build chapters are largely under construction, and Part III is ahead. Two planning consequences. First, the course is best consumed progressively, following the repository, rather than blocked out as a finished curriculum on a calendar. Second, the finished early material is genuinely useful today: Part I alone is one of the cleaner concise theory references for inference available free, in either language.

What is RadixAttention and why does it anchor the course?

RadixAttention is SGLang’s signature design choice, and the course gives it a dedicated build chapter, which tells you the course is about SGLang rather than generic serving. The idea: prefix sharing is common in real traffic, multi-turn conversations, few-shot templates, agent loops re-reading the same context, and SGLang stores the KV cache in a radix tree so any request whose prefix matches previously computed tokens reuses that computation instead of recomputing it. The reuse is automatic rather than declared, which is why the engine’s prefix-caching numbers surprise people who benchmark it naively. Building a small version in Part II, after implementing the plain KV cache, is structured exactly right: you feel the cost the radix tree eliminates before you build the structure that eliminates it.

Can you take this course without a GPU?

Partially, and the course is explicit about which parts. Part I is written to be completed with no code and no GPU, covering the conceptual spine from Transformer basics through KV-cache memory arithmetic. A cloud rental covers Part 0’s deployment chapter cheaply, since Qwen3-0.6B is small enough that a modest instance serves it. Part II is the GPU-dependent stretch: the build chapters run real inference loops, and while emulation exists in theory, the memory and scheduling lessons depend on observing real hardware behavior. The pragmatic no-GPU plan is Part I now, bookmarking the build chapters, and treating the deployment chapter as the trigger to rent an hour of compute when you are ready. Learners who stretch the no-GPU phase too long tend to fossilize the theory without the corrective of real measurements, so set yourself a deadline for the first deployment, even a small one, because the theory holds together better once a single real server has grounded it.

How does the Datawhale partnership shape the course?

Datawhale runs some of the largest open study groups in the Chinese-speaking ML community, with a track record of turning dense material into cohort-friendly courses, and that pedigree shows in the structure: short chapters, explicit status tracking, bilingual editions, and a syllabus organized for study groups rather than solo reference browsing. For an international learner the partnership mainly means two things. Cohort pacing, where chapters open on a schedule and community discussion accompanies them, may exist in Chinese channels you will not join, so plan for self-paced study. And the bilingual commitment means English editions follow Chinese ones, so checking the language badge per chapter, rather than assuming one or the other, is the habit that saves disappointment. What the partnership does not mean is a quality compromise on the engineering side: the syllabus tracks SGLang’s real subsystems, not a simplified curriculum, which is the difference between a community course about a project and a project course hosted by a community.

What should you read alongside the course?

The course is self-contained for theory, but two companions pay for themselves. A from-scratch MLX-side build course makes a good counterweight: it covers the same mechanisms, batching, paging, scheduling, from a different engineering culture, and seeing two implementations of the same idea is the fastest way to separate the essential from the incidental. And an operator-side field guide to the serving-engine landscape keeps the practical context in view while the course goes deep on one engine, because the goal is judgment about engines generally, not loyalty to one. Read the course chapters first, the companions when a concept needs a second angle, and the SGLang documentation itself once Part III starts pointing at it. One caution for the impatient: skipping ahead into SGLang’s issue tracker before Part I is a reliable way to mislearn, because tracker discussions assume exactly the vocabulary this course exists to hand you.

Does this course replace reading SGLang source code?

No, and it is designed knowing that. Reading a production engine cold is inefficient because scale hides structure: the core ideas sit under layers of hardening, fallbacks, and platform branches that exist for reasons a newcomer cannot see. The course front-loads the structure, an engine’s modules, the request lifecycle, the cost models, so that a source dive afterward lands on prepared ground. Part III then walks specific internals, attention backends, CUDA Graph integration, quantization paths, that would each take days to find unguided. The efficient sequence is course first, then targeted source reading with the course’s architecture map in hand. Contributors get an extra benefit: the course teaches the project’s own vocabulary, so issues and pull-request discussions read faster after it.

What do TTFT, TPOT, and goodput mean once you can compute them?

The benchmark chapter earns its place because these metrics stop being jargon the moment you measure them under load. Time to first token is what a user perceives as responsiveness, and it is dominated by prefill compute. Time per output token governs the streaming feel of generation and is dominated by memory bandwidth and batch composition. Goodput is the metric teams forget: throughput counted only for requests that met their latency targets, which is the number that collapses when a system is tuned for raw throughput at any cost, and the metric the course teaches you to design an experiment around, because a benchmark that does not declare its targets and percentiles is a slogan rather than a measurement. The course teaches the percentile view on top, p50 versus p99, because averages hide exactly the tail behavior that makes users call a system slow. After this chapter, capacity planning conversations change register: you stop asking how fast the server is and start asking which percentile of which metric, at what concurrency, under what batch mix.

Start Part 0 tonight if a GPU is nearby, read Part I this week regardless, and follow the repository for the build chapters rather than waiting for a finished release. Zero to SGLang is the rare official course that respects both the beginner and the contributor, and the per-chapter status table makes it usable at every stage of its own construction.