7 Best Foundational LLM & AI Agent Papers: The Reading List That Actually Matters

These are the seven papers I’d hand a new hire before they touch an agent codebase. Not the full ML canon, just the papers whose vocabulary shows up in every framework README, every product launch, every LangGraph tutorial. Read them in order and most “new” agent content on social media starts to look like a re-skin of one of these.

Why these seven AI agent papers

There are thousands of arXiv papers on large language models. The signal-to-noise ratio is brutal. This roundup filters to the AI agent papers that earned a permanent place in the working vocabulary of agent engineering, the ones a senior engineer will reference in passing assuming you’ve read them, and the ones that, once you’ve read them, make framework documentation legible instead of opaque.

I’ve kept it to seven on purpose. A longer list turns into a backlog you never start. Seven is a weekend. Each entry below gets a two-paragraph honest summary and one explicit “why it matters” line, then a direct link to the canonical version on arXiv or NeurIPS. No mirrors, no paywalls.

If you want the applied counterpoint to these papers (runnable code that implements each pattern), the GenAI Agents and Hugging Face Agents Course spotlights are the right next stop. This reading list is the conceptual layer underneath them.

1. ReAct: Where the agent loop gets its name

Yao et al., 2022. The AI agent paper that named the pattern most production agents still use: interleave a reasoning trace with tool-calling actions, so the model observes a result, reflects in plain text, and decides the next step in a single loop. Reasoning without action stalls; action without reasoning flails. ReAct is the surgical splice of both.

Every framework that calls itself an “agent framework” (LangGraph, CrewAI, AutoGen, Agno) implements some variant of this loop under the hood. Read this paper once and the rest of the agent literature stops being mysterious.

Why it matters: it’s the conceptual primitive underneath every “thought → action → observation” framework. Skipping it means learning the same idea second-hand through framework abstractions.

↔ Read ReAct on arXiv (2210.03629)

2. Generative Agents: The sandbox that proved memory works

Park et al., 2023. The AI agent paper runs twenty-five LLM-driven agents inside a Smallville sandbox with memory, reflection, and daily plans. The now-famous result: a Valentine’s Day party self-organized across agent-to-agent invitations without any script. It was the first convincing public demo that the memory + reflection + planning triplet produces emergent, believable social behavior.

The architecture in the paper (observation, retrieval, reflection, planning) is still the reference design for character agents, simulation agents, and any “persona-driven” product. When someone says “agent memory,” this is the mental model they’re operating from.

Why it matters: the canonical existence proof for agent memory as a first-class subsystem, not a side feature. Everything since (Mem0, Zep, Letta) refines this skeleton.

↔ Read Generative Agents on arXiv (2304.03442)

3. Toolformer: Models that decide when to call a tool

Schick et al., NeurIPS 2023. The AI agent paper that taught LLMs to self-supervise tool-use decisions, to insert the right tool call at the right position in a generation, on their own, without hand-coded triggers. The tools in the paper are simple (calculator, QA retriever, search, calendar, translation) but the implication is large: tool selection doesn’t need a hard-coded router.

If you’ve ever used OpenAI function-calling or Anthropic tool-use and wondered where the design came from, this is the paper. It’s the empirical case that a model can learn tool composition as a language behavior.

Why it matters: the direct ancestor of every modern function-calling API. Read it and you’ll stop being surprised by what the model decides to invoke.

↔ Read Toolformer on NeurIPS Proceedings (2023)

4. Chain-of-Thought Prompting: The two-word win

Wei et al., 2022. The paper that documented, with benchmarks, what every user had stumbled on: appending “let’s think step by step” to a prompt lifts multi-step reasoning scores on tasks like GSM8K by double-digit percentages. The cost is two cents; the gain is one of the largest in the prompt-engineering literature.

It’s also the conceptual underpinning of every later reasoning paper. Tree of Thoughts, Reflexion, Self-Consistency, and the entire “reasoning model” product category. If you want to understand why reasoning models behave the way they do, the trail starts here.

Why it matters: the highest leverage-to-cost ratio in prompt engineering. And the foundation of the entire “reasoning” research line that followed.

↔ Read Chain-of-Thought Prompting on arXiv (2201.11903)

5. Tree of Thoughts: When the first path isn’t the right one

Yao et al., 2023. The AI agent paper that generalizes Chain-of-Thought into a search problem: the model explores multiple candidate reasoning branches, evaluates intermediate states, and backtracks when a branch looks weak. For tasks where the first plausible path is wrong (creative writing under constraints, multi-step math puzzles, 24-game), the accuracy lift over CoT is often 20+ points.

It’s more expensive than CoT in tokens, so the right framing is “CoT by default, ToT when CoT fails on tasks with verifiable intermediate progress.” That trade is the actual engineering decision.

Why it matters: the cleanest paper-of-record for “search over reasoning”, the move you reach for when you’ve exhausted linear prompting.

↔ Read Tree of Thoughts on arXiv (2305.10601)

6. Reflexion: The cheapest win in the agent literature

Shinn et al., NeurIPS 2023. This AI agent paper is one trick: after a failed attempt, the agent writes itself a short verbal note about what went wrong, then retries. On HumanEval and AlfWorld the gains are typically 10-20 points over the same agent without the reflection step, for the cost of one extra generation per failure.

It’s the agent equivalent of “rubber-duck debugging” and it generalizes: wrap almost any existing agent in a reflection loop and you’ll usually see accuracy go up. Of the seven papers here, this is the one you can ship to production on Monday morning.

Why it matters: the highest-ROI agent pattern in the list. Tiny code change, measurable benchmark gain, no new infra.

↔ Read Reflexion on NeurIPS Proceedings (2023)

7. RAG Survey: The shared vocabulary for retrieval

Gao et al., 2023. The taxonomy paper, and the shared vocabulary the other AI agent papers assume for retrieval. Pre-retrieval, retrieval, post-retrieval. Sparse versus dense retrievers. Naive RAG versus modular RAG. The paper organizes 150+ references into a single mental model so that when a teammate says “we’re doing RAG,” you have a shared language for which RAG they mean.

If the first six papers are about reasoning, this one is about the other half of most production agent stacks: retrieval. Read it before you design your retrieval pipeline; the design space is smaller than it looks once the taxonomy is in your head.

Why it matters: cuts RAG design conversations from an hour to ten minutes by giving the team a shared map. The single most cited survey in the retrieval-augmented generation literature.

↔ Read the RAG Survey on arXiv (2312.10997)

Why is ReAct the paper every agent implements?

ReAct is the paper that named the agent loop, which interleaves reasoning and acting: the model thinks about what to do, takes an action, observes the result, thinks about what to do next, and repeats. The paper showed that the interleaving is what makes the agent work, because a model that only reasons gets stuck and a model that only acts gets lost, and the interleaving is what makes the agent able to recover from a wrong action by thinking about the next one.

Every modern agent is a ReAct agent with extra steps. The extras are usually memory, tool selection, and planning, but the core loop is the same as the paper described in 2022. A team that has not read the paper is going to spend time rediscovering the loop, and a team that has read the paper is going to spend that time on the extras. The paper is short, the loop is implementable in a few hundred lines of code, and the lessons transfer to every agent the team is going to build.

What did Generative Agents prove that earlier agent papers had not?

Generative Agents proved that an agent with memory and reflection can produce emergent social behavior that the authors did not script, and the proof was a sandbox of 25 agents in a Sims-like world who woke up, formed relationships, threw a Valentine’s Day party, and behaved like a small town. The agents did not have any of this scripted; they had a memory of their observations, a reflection step that wrote down higher-level thoughts, and a planning step that turned the thoughts into actions.

The paper is also a practical lesson in what an agent needs to feel coherent: a memory that is queried by relevance, a reflection step that runs at appropriate intervals, and a planning step that balances the agent’s own goals against the social context. A team that wants agents that feel coherent rather than scripted is going to end up implementing something close to the paper’s architecture.

Why did Toolformer matter more than it gets credit for?

Toolformer was the paper that showed a language model could learn to call external tools on its own, by training on a corpus where the tools were inserted at the right points and the model was rewarded for using them when they helped and not using them when they did not. The paper showed that the model can decide when a tool is useful, which is the part of the agent loop that prompt-only approaches could not solve reliably.

The paper is also a useful lesson in what the alternative looks like. Before Toolformer, agents were mostly prompt-only, which meant the model was told when to call a tool rather than deciding for itself, and the prompting was brittle. After Toolformer, the field took the lesson and built agents that let the model decide, and the difference shows up in agents that look natural and agents that look mechanical.

Why is Chain-of-Thought the highest-impact paper in the entire list?

Chain-of-thought prompting is the paper that showed that asking the model to think step by step before it answers improves the answer, and the improvement is consistent across model sizes, across tasks, and across domains. The two-word instruction, think step by step, is the cheapest win in the entire field, and the paper documented it rigorously enough that no serious agent system ignores it.

The paper is also a useful lesson in what the boundary of the technique is. Chain-of-thought prompting helps when the task is the kind of task where thinking helps: math, logic, multi-step reasoning, and structured planning. It does not help, and can hurt, when the task is the kind of task where the model already knows the answer: factual recall, simple classification, and one-step lookups.

Why is Reflexion the cheapest win in the agent literature?

Reflexion is the paper that showed that an agent that reflects on its own failures and writes the reflection into memory does measurably better on the next attempt, and the implementation is a few lines of code: take the trajectory, ask the model what went wrong, write the answer into the agent’s memory, and let the agent try again. The win comes from the model being able to articulate its own failure mode, which is a thing modern models are surprisingly good at, and from the memory being available on the next attempt.

The paper is a useful lesson in what the minimum viable agent looks like. A team that has not read the paper is going to spend a lot of time on elaborate memory and planning systems, and a team that has read the paper is going to ship the simple version first and only add the elaborate parts when the simple version stops being enough. The simple version is the right starting point.

How to read them

Don’t read all seven AI agent papers in one sitting. Read ReAct and Chain-of-Thought first, they’re short and they open the rest. Then Toolformer and Reflexion, because they’re the ones you’ll act on this week. Save Generative Agents and Tree of Thoughts for a longer weekend read; they’re denser but stay with you. Read the RAG Survey when you actually need to design retrieval, out of context it’s a reference, in context it’s a cheat sheet.

Each paper links to its canonical arXiv or NeurIPS version. No mirrors, no redirects, no paywalls. If you want runnable code that implements these patterns, the GenAI Agents repository and the Hugging Face Agents Course are the applied companions.

Seven papers, one weekend. ReAct, Generative Agents, Toolformer, Chain-of-Thought, Tree of Thoughts, Reflexion, and the RAG Survey: the foundational reading list that turns “I’ve heard of agents” into “I’ve read the source.” Bookmark this page; you’ll come back to it.