Google Agent Whitepaper Explained: The 2026 Guide

The Google agent whitepaper, published in September 2024 under the plain title Agents, is the document most builders point to when they need a clean definition of what an AI agent actually is. Written by Julia Wiesinger, Patrick Marlow, and Vladimir Vuskovic, it skips the hype and lays out a compact cognitive architecture: a language model, a set of tools, and an orchestration layer that loops between them. If you have read vendor blog posts that contradict each other on what counts as an agent, this whitepaper is the shared vocabulary that sorts them out.

This guide unpacks the Google agent whitepaper in plain English, then connects it to a companion document Google released later. The goal is not to summarize the paper line by line. It is to explain the mental model the paper teaches, so you can read other agent framings, from Anthropic, OpenAI, and the research literature, with a stable reference point.

Why this whitepaper became the reference everyone cites

Before late 2024, the word agent was doing too much work. Some people used it to mean a chatbot with memory. Others meant a script that calls an API. A third group meant a system that plans multi step tasks on its own. The Google agent whitepaper tightened the definition without narrowing it so much that it excluded real systems, which is why it gets cited in engineering blogs, course syllabi, and follow on papers more than almost any other vendor document on agents.

The paper is short, roughly forty pages with figures, written for developers rather than executives. It assumes you know what a large language model is and that you have called an API before. From that baseline it builds the agent concept from three parts, shows how they talk to each other, and then spends most of its pages on tools, the part that matters most in production.

The definition, in the whitepaper’s own words

The paper defines a Generative AI agent as an application that attempts to achieve a goal by observing the world and acting upon it using the tools it has at its disposal. Two adjectives do the heavy lifting in that sentence. The agent observes, meaning it takes in information from its environment rather than only from a static prompt. And it acts, meaning it can change something outside itself, like writing to a database or sending an email, not just generate text.

The paper also stresses two behavioral properties. Agents are autonomous: they can act without a human stepping in at every step, especially with a clear objective. And they are proactive: even without an explicit instruction set, an agent can reason about what to do next. A bare language model does neither. It answers one prompt, then waits.

The cognitive architecture in the Google agent whitepaper

This is the heart of the Google agent whitepaper. The paper calls the combination of an agent’s core components its cognitive architecture, and it names exactly three. Everything else in the paper, the reasoning techniques, the tool taxonomy, the production examples, hangs off these pillars.

The first pillar is the model. In this framing the model is the language model that acts as the centralized decision maker for the agent. It can be one model or several, small or large, general purpose or fine tuned. The key requirement is that it can follow instruction based reasoning frameworks like ReAct, Chain-of-Thought, or Tree-of-Thoughts. The paper makes a point worth pinning down: the model is usually not trained with the specific tool choices or orchestration setup of the agent. Those are configured at runtime around it.

The second pillar is tools. Foundational models, however good at text and image generation, cannot interact with the outside world on their own. Their knowledge stops at their training cutoff, and they have no hands. Tools bridge that gap. The paper notes that tools usually map onto common web API methods like GET, POST, PATCH, and DELETE, and that they let an agent support techniques like retrieval augmented generation. We will come back to the three tool types Google defines, because that taxonomy is the most reused part of the paper.

The third pillar is the orchestration layer. This is the cyclical process that governs how the agent takes in information, performs internal reasoning, and uses that reasoning to choose its next action. The loop runs until the agent reaches its goal or a stopping point. The complexity varies wildly: some loops are simple calculations with decision rules, others chain together logic, extra machine learning algorithms, and probabilistic reasoning. This is the layer where memory, state, and planning live.

The cognitive loop, told through a chef

To make the orchestration layer concrete, the Google agent whitepaper reaches for a kitchen analogy. Picture a chef in a busy restaurant. The chef’s goal is to turn orders into finished dishes. First they gather information: the patron’s order, what is in the pantry and refrigerator. Then they reason internally about which dishes and flavor profiles they can build from those ingredients. Then they act: chopping vegetables, blending spices, searing meat. At every stage they adjust, refining the plan as ingredients run low or as feedback comes back from the pass. That cycle of intake, planning, execution, and adjustment is the cognitive architecture.

The paper then maps that cycle onto reasoning frameworks. ReAct gives the model a thought process for reasoning and acting on a query. Chain-of-Thought enables reasoning through intermediate steps, with sub techniques like self consistency, active prompt, and multimodal Chain-of-Thought. Tree-of-Thoughts generalizes Chain-of-Thought and suits exploration or strategic lookahead, letting the model explore several thought chains before committing. The paper walks through a sample ReAct trace where the agent emits a question, a thought, an action, an action input, an observation, and a final answer, repeating the thought and action steps as many times as needed.

The reason this matters is practical. When an agent grounded in ReAct faces a question it cannot answer from training data, instead of guessing and hallucinating, it picks a tool, calls it, reads the result, and only then answers. The Google agent whitepaper shows this with a flights example: the model could have fabricated a flight time, but it used a Flights tool to pull real-time data and summarized that instead.

Agents versus models, side by side

One of the most copied parts of the Google agent whitepaper is a simple comparison between models and agents. The contrast is worth remembering because it explains why bolting a prompt onto a model does not make an agent.

A model’s knowledge is limited to its training data. An agent’s knowledge is extended through tools that connect to external systems. A model produces a single inference from a user query and manages no session history unless you build that yourself. An agent manages session history, which lets it do multi turn inference informed by earlier decisions. A model has no native tool implementation. An agent has tools built into its architecture. And a model has no native logic layer, so users construct reasoning prompts by hand. An agent ships with a cognitive architecture that already uses frameworks like ReAct or pre built stacks like LangChain.

The shorthand the paper keeps returning to is that an agent is a program that extends beyond the standalone capabilities of a Generative AI model. The model is the brain. The tools are the hands and eyes. The orchestration layer is the habit of thinking before acting and revising after. Strip any one of the three away and you are back to a model with a prompt.

The three tool types: extensions, functions, data stores

This is the section of the Google agent whitepaper that practitioners cite most, because the taxonomy resolves a lot of confusion about how agents reach the outside world. Google defines three primary tool types its models can interact with, and gives each a distinct execution location and use case.

Extensions are the closest thing to plug and play. An extension bridges the gap between an agent and an API in a standardized way, teaching the agent through examples how to use an endpoint and what parameters it needs. Extensions execute agent side, meaning the agent makes the live API call itself. The paper’s running example is flight booking: a user says they want to fly from Austin to Zurich, and instead of fragile custom code that tries to parse the origin and destination, an extension lets the agent learn the Google Flights API from examples and call it. Google ships ready made extensions through a hub, including a Code Interpreter that generates and runs Python from a natural language description.

Functions flip the execution to the client side. The model still decides which function to call and what arguments to pass, but it does not make the live API call. It outputs a structured payload, the paper shows a JSON object with a function name and arguments, and the developer’s client application runs the actual call. This split matters when security or authentication rules keep the agent off the API, when timing or order of operations rules out real-time calls, when extra data transformation is needed, or when the team wants to stub APIs during development. The ski trip example is the clearest illustration: a user asks where to take a family ski trip, and instead of returning a bullet list of cities that is hard to parse, the model emits a structured function call with the city names so downstream code can fetch images through the Google Places API.

Data stores solve the knowledge problem. The paper likens a language model to a vast library that never acquires new books. Data stores fix that by giving the agent a vector database of fresh, structured or unstructured content. The store turns documents, websites, PDFs, spreadsheets, or database rows into vector embeddings, and at query time the agent embeds the user’s question, matches it against the store with an algorithm like SCaNN, retrieves the relevant chunks, and folds them into its reasoning. This is the mechanism behind retrieval augmented generation, and the paper walks through a five step RAG loop from query to final response.

The paper summarizes the three with a clean rule of thumb. Extensions run agent side and fit when you want the agent to own the API interaction, especially with pre built extensions. Functions run client side and fit when security, timing, or control concerns keep the agent off the API. Data stores run agent side and fit when the goal is grounding the model in fresh private knowledge.

Targeted learning: how the model gets better at picking tools

The Google agent whitepaper spends a section on a question that surfaces in every production deployment: how do you make the model reliably choose the right tool? The paper frames three approaches, again through the cooking analogy. In-context learning hands the model a prompt, a set of tools, and a few examples at inference time, so it figures out on the fly how and when to use each tool. ReAct is the natural language example of this. Retrieval based in-context learning dynamically pulls the most relevant tools and examples from an external memory at runtime, like the Vertex AI Example Store or the data store RAG setup. Fine tuning based learning trains the model on a dataset of specific examples ahead of time, so it already knows when and how to apply a tool before any user query arrives.

The paper is careful not to declare a winner. Each approach trades off speed, cost, and latency differently, and the practical move is to combine them. In-context learning is fast to try, retrieval adds freshness without retraining, and fine tuning locks in stable behavior for high volume tool use.

From prototype to production: the companion document

The Google agent whitepaper closes by pointing from concepts to product. It shows a LangChain and LangGraph quick start that wires a gemini model to a search tool and a Google Places tool to answer a two part question about a football game and a stadium address, then turns to Vertex AI agents as the managed path to production. On Vertex AI, developers define goals, task instructions, tools, sub agents for task delegation, and examples through a natural language interface, and the platform handles evaluation, debugging, and the infrastructure around it.

This is where the companion document comes in. Google followed the original whitepaper with an Agent Companion, hosted alongside it, that takes the same architecture and applies it to shipping agents in practice. Where the whitepaper defines the model, tools, and orchestration loop, the companion is more operational, walking through how to scope an agent, pick tools that fit the task, evaluate agent behavior, and avoid the common failure modes that appear once you leave the prototype behind. Reading the two together gives you the mental model and the playbook. The whitepaper is the theory you need to reason about any agent framing. The companion is the engineering checklist you need to ship one.

How Google’s framing compares to Anthropic and OpenAI

The Google agent whitepaper is one of three widely cited vendor framings, and it is worth seeing where it sits relative to the others, because the differences are about emphasis rather than contradiction.

Google leads with architecture. The model, tools, orchestration triad is front and center, and the orchestration loop is treated as the load bearing component. Anthropic’s Building Effective Agents guide, by contrast, leads with workflow patterns. It distinguishes single step workflows from agent loops and spends most of its energy on when an agent is overkill, arguing that many production needs are better served by a fixed chain of prompt calls than by an autonomous loop. Where Google’s paper gives you the parts of an agent, Anthropic’s gives you the discipline to decide whether you need one at all.

OpenAI’s practical guide to building agents, published as a business PDF, sits closer to a build checklist. It focuses on identifying candidate tasks, designing the tool set, writing the right instructions, and scaffolding an orchestration loop with its Agents SDK. It shares Google’s model plus tools plus loop skeleton but spends less time on the theory of the loop and more on the recipe for shipping.

So a useful reading order is this. Start with the Google agent whitepaper for the shared vocabulary and the cognitive architecture. Then read Anthropic’s guide for the decision discipline of when to escalate from a workflow to a true agent. Then read OpenAI’s guide for the production scaffolding. All three assume the same underlying loop. They differ in which question they answer first.

Who should read it, and how to use it

The Google agent whitepaper is the right first read for engineers and technical leads about to build an agent and wanting a shared mental model before picking a framework. It is also useful for anyone evaluating agent claims, because the model plus tools plus orchestration test exposes quickly whether a product labeled agent is actually an autonomous loop or just a prompt wrapper. Read the original on Kaggle for the figures, then skim the companion for the operational version.

The paper is openly accessible and short enough to read in one sitting, and its definitions have aged well. The taxonomy of extensions, functions, and data stores still maps cleanly onto modern tool calling, function calling, and retrieval augmented generation APIs across vendors. That durability is the best argument for reading it before the framings that came after.

The Google agent whitepaper is the canonical definition of an AI agent as a model plus tools plus an orchestration loop, and it is the reference point to read before any vendor-specific build guide. Read it for the vocabulary and the cognitive architecture, then pair it with the companion document for the production playbook. Read the Agent Whitepaper and the Agent Companion on Kaggle.