Building Effective Agents: Anthropic’s Guide Explained

Building Effective Agents is the December 2024 engineering essay from Anthropic that quietly became the most cited agent architecture reference of the generative AI era. It is short, free, and intentionally anti hype. Instead of selling a framework, it splits agentic systems into two categories, workflows and agents, and then enumerates five workflow patterns that you can build in a few hundred lines of code against any LLM API. This deep dive explains what the guide actually says, why its building effective agents thesis has aged so well, and where it overlaps or diverges from the Google agent whitepaper and Andrew Ng’s four design patterns.

The whole essay rests on one distinction stated in the first section. Anthropic divides agentic systems into workflows, where LLMs and tools are orchestrated through predefined code paths, and agents, where the LLM dynamically directs its own process and tool usage, maintaining control over how it accomplishes the task. That single sentence reframes a debate that the wider field was having in confusing terms. Most of what people called agents in 2024 were actually workflows, and Anthropic argues that is a good thing. Workflows are predictable, cheap, and testable. Agents are powerful but expensive and prone to compounding errors. Knowing which one you are building is the first decision, and most teams should pick a workflow.

The Building Block: The Augmented LLM

Before any pattern, Anthropic defines the atom. The building block of agentic systems is the augmented LLM, an LLM enhanced with retrieval, tools, and memory, wrapped in a loop that lets it iterate until a stopping condition is met. Every pattern in the guide is a different way of composing these augmented LLMs. The emphasis matters because it locates the hard problem. The hard problem is not wiring LLMs together. The hard problem is making the augmented LLM itself reliable, which means good retrieval, well designed tools, and prompts that hold up under the failure modes the loop will expose.

Prompt Chaining: Steps In A Fixed Sequence

The first workflow pattern is prompt chaining. You break a task into a sequence of consecutive steps, where each LLM call processes the output of the previous one. Programmatic or rule based gates can be inserted between steps to decide whether the chain should continue, retry, or abort. The classic example in the guide is generating a document: first produce an outline, run a programmatic check that the outline meets a fixed set of criteria, then pass the validated outline to a second prompt that writes the full document.

The tradeoff is that the task has to decompose into predictable subtasks with a fixed order. Where that holds, prompt chaining gives you reliability and latency control that a freeform agent cannot match. Each step runs on a prompt tuned for one job, you can swap a smaller model into the cheap steps, and a gate that fails fast saves the cost of the rest of the chain.

Routing: Classify Then Dispatch

Routing classifies an input and sends it to a specialized downstream handler. The point is separation of concerns. Each downstream branch runs a prompt tailored to one input type, so you never have to write a single mega prompt that tries to handle every case and degrades on all of them. The guide’s example is a customer service system: an initial LLM classifies the query as a refund question, a technical issue, or a trivial FAQ lookup, then routes each to a dedicated prompt or model. Trivial queries can go to a smaller, cheaper model while complex ones go to a more capable one.

Routing pays off when your inputs fall into a small, stable set of categories that benefit from being handled separately. It fails when categories overlap or when the classifier itself is unreliable, because a misrouted input lands in a prompt that was never designed for it.

Parallelization: Sectioning And Voting

Parallelization runs multiple LLM operations at the same time and aggregates their outputs programmatically. Anthropic splits it into two forms. Sectioning breaks a task into independent subtasks that run concurrently, then combines them, useful when a long document can be summarized in chunks and the chunks stitched back together. Voting runs the same task multiple times to get diverse outputs, then picks or merges, useful when you want a model to vote on whether a piece of content is safe or to generate several candidate translations and keep the best.

The guide’s flagship use case for sectioning is guardrails: one model instance screens the user query for inappropriate content while another simultaneously generates the response, so the screen runs in parallel rather than blocking the reply. Parallelization trades extra token cost and added latency coordination for either throughput or robustness, depending on whether you section or vote.

Orchestrator Workers: Dynamic Decomposition

Orchestrator workers is the pattern that comes closest to feeling like an agent while still being a workflow. A central orchestrator LLM dynamically decomposes a task, assigns the subtasks to specialized worker LLMs, and synthesizes their results into a final answer. The distinction from prompt chaining is that the subtasks are not predetermined. The orchestrator decides what workers are needed based on the specific input.

The canonical example is a coding tool that needs to make coordinated changes across several files. Which files, and how many, depends on the change being requested, so a fixed chain cannot capture it. The orchestrator inspects the request, plans the file level edits, dispatches each to a worker, and merges the diffs. This pattern fits tasks where the work is structurally variable but the synthesis step is well defined.

Evaluator Optimizer: The Refinement Loop

In the evaluator optimizer pattern, one LLM generates a response and a second LLM evaluates it and provides feedback, then the generator revises in an iterative loop. This is the workflow analogue of the reflection pattern Andrew Ng teaches, with the crucial difference that here the evaluator is a separate LLM with explicit evaluation criteria, not the same model grading itself.

The guide’s example is literary translation. A first model translates a passage. An evaluator model that knows the source language critiques the translation for missed nuance, awkward register, and dropped meaning. The translator revises. The loop continues until the evaluator is satisfied. This pattern earns its complexity only when you have clear evaluation criteria and you can demonstrate that the generator measurably improves under critique. Without an explicit rubric, the evaluator is just agreeing with the generator.

The Agent: Autonomy When You Cannot Predict The Steps

After the five workflows, Anthropic describes the agent itself. An agent is an LLM operating autonomously in a loop, using tools, observing the results, and deciding the next action based on that feedback. It can pause for human input, and it terminates when it reaches a stopping condition, which may be task completion, a step budget, or an error it cannot recover from.

Anthropic is careful about when this is the right choice. Agents are recommended for open ended problems where it is difficult or impossible to predict the required number of steps in advance. They shine on tasks like research synthesis, open ended coding, or open ended exploration where the path is genuinely discovered as the work proceeds. They are also, by Anthropic’s own framing, more expensive and more brittle than the workflow that could have done the job, so reaching for an agent is a deliberate escalation, not a default.

When Not To Reach For Building Effective Agents

The most quoted line in the essay is the caution against overbuilding. Anthropic states plainly that agentic systems often trade latency and cost for better task performance, and that this trade is only worth it when the complexity buys you something. Their explicit guidance: optimizing single LLM calls with retrieval and in-context examples is usually enough. If a well prompted LLM with good retrieval solves your problem, you do not have an agent problem, you have a prompting and retrieval problem.

This is the section most teams skip and most production failures trace back to. The guide lists the failure modes that come from premature agent adoption. Higher cost, because every tool call and loop iteration burns tokens. Higher latency, because the loop has to complete. Compounding errors, because an agent that takes a wrong step early reasons about a poisoned context for the rest of the run. And reduced predictability, which makes evaluation, the thing production actually runs on, much harder. The advice is to start with the simplest solution possible and only add complexity when you can point to the specific limitation that the complexity removes.

Augmented LLMs And Why Anthropic Dislikes Frameworks

The second persistent theme in the essay, alongside simplicity, is skepticism toward heavy agent frameworks. Anthropic reports that the most successful implementations they observed were not using complex frameworks but simple, composable patterns. The argument is that frameworks hide the loop, the prompts, and the tool definitions behind abstractions, and that the teams shipping reliable agents needed direct control over exactly those things. This is the same bet Andrew Ng makes in his framework free course, taught from a different angle, and it is the throughline of the guide’s appendix on tool design.

The Appendix: Agent Computer Interface

The essay closes with two appendices, and the second is the one that practitioners return to. It is on prompt engineering your tools, and it introduces the idea of the Agent Computer Interface, the ACI, as a peer to the human computer interface. The claim is that the effort you put into how an agent sees and calls its tools matters as much as the effort you put into a UI for human users, and that bad tool design is a leading cause of agent failure.

The concrete recommendation is a Japanese engineering term, poka yoke, which means mistake proofing. Tools should be shaped so that the model is unlikely to misuse them. The example given is parameters that accept file paths: prefer requiring absolute paths over relative paths, because a relative path resolved against the wrong working directory is a silent and common error. Keep parameter formats simple and consistent, keep the number of tools small, and group related parameters so the model can reason about them as a unit. This appendix is short and worth reading in full because it converts an abstract idea, design your tools well, into a checklist you can apply.

How It Compares To The Google Whitepaper And Ng’s Patterns

Place this guide next to the Google agent whitepaper and Andrew Ng’s four design patterns and a useful map emerges. All three converge on the augmented LLM as the atom and on the tool and retrieval loop as the engine. They diverge on what they choose to enumerate. Anthropic enumerates five workflow patterns plus the agent, and spends most of its words telling you to use the simplest one. Google’s whitepaper enumerates the cognitive architecture, reasoning, memory, tools, and the Model Context Protocol, and spends most of its words on the substrate. Ng enumerates four patterns, Reflection, Tool Use, Planning, and Multi Agent, and spends most of his words on the evaluation discipline that makes any of them ship.

Anthropic’s evaluator optimizer and Ng’s reflection are the same idea seen from two angles. Anthropic’s orchestrator workers and Ng’s multi agent overlap heavily, with the Building Effective Agents guide framing it as a workflow with dynamic decomposition and Ng framing it as a pattern for communication and handoff. Where Building Effective Agents is unique is in the routing and parallelization patterns, which neither Google nor Ng foreground, and in the relentless simplicity thesis that makes the guide so quotable. If you read only one of the three, read Building Effective Agents for the patterns and the discipline, then read Google for the substrate and Ng for the evaluation rigor.

The Google whitepaper frames the same territory through a cognitive architecture lens. Where Building Effective Agents says augmented LLM, Google says Model plus Tools plus an orchestration layer that runs a perception to planning to action loop. The vocabulary difference matters less than the emphasis. Google spends its pages on the substrate, the reasoning techniques like chain of thought and ReAct and tree of thoughts that sit inside the Model box, and the tool types like extensions, functions, and data stores that the Model calls out to. Building Effective Agents barely mentions any of that substrate and instead spends its pages on which wiring pattern you should pick. The two are complements, not substitutes. Read Google to understand what is inside the box, then read Building Effective Agents to decide how many boxes you need and how to connect them.

Ng’s four patterns are the third vertex. Reflection is the loop where the model critiques and revises its own output, which maps directly onto the evaluator optimizer pattern in Building Effective Agents once you accept that the generator and the evaluator do not have to be the same model. Tool use is the capability that makes every other pattern productive, and Ng treats it as a pattern in its own list where Building Effective Agents folds it into the augmented LLM baseline. Planning is the pattern Ng foregrounds and Building Effective Agents treats with the most suspicion, because a planning step that decomposes a goal into subtasks is exactly the kind of autonomy that fails silently when the decomposition is wrong. Multi agent is Ng’s name for what Building Effective Agents calls orchestrator workers, and again the disagreement is about emphasis, with Ng interested in the communication protocols between agents and the Building Effective Agents essay interested in whether you should have more than one at all. The honest summary is that the three sources describe the same design space from three angles, and a practitioner who internalizes all three gets a far sharper sense of when each pattern pays off than any single source provides. Read them in that order, Google for the substrate, Building Effective Agents for the patterns, Ng for the evaluation rigor, and you will notice that the disagreements between them are almost always about emphasis and risk tolerance rather than about what an agent fundamentally is.

Read Building Effective Agents for the vocabulary and the restraint. The five workflow patterns, prompt chaining, routing, parallelization, orchestrator workers, and evaluator optimizer, plus the autonomous agent, give you a complete map of what you can build before you reach for anything autonomous. The real lesson is the caution: optimize single LLM calls with retrieval and in-context examples first, build the simplest workflow that solves the problem, and only escalate to an agent when you can name the limitation that the extra complexity removes. That thesis is why this short essay still anchors the field two years on. Read Building Effective Agents on Anthropic.