Imagine an AI that doesn't just respond to your questions but actively pursues goals, remembers past interactions, learns from mistakes, and makes reasoned decisions about what to do next. This isn't science fiction—it's the emerging frontier of language agents.
When you ask ChatGPT a question, it delivers an answer and the conversation ends. But what if that same underlying technology could instead navigate a website to book your vacation, control a robot to clean your kitchen, collaborate with other AIs to solve complex problems, and remember your preferences for next time? This shift—from passive language model to active language agent—represents one of the most significant transformations in artificial intelligence today.
The problem is that we're building these agents without a shared blueprint. Researchers use different terms for the same concepts, design agents that can't be easily compared, and often reinvent the wheel. Some agents can reason about problems but can't remember past experiences. Others can learn but don't plan ahead. The field lacks a common language for describing what these systems do and how they do it.
The Hidden Structure Inside Smarter AI Systems
What makes a language model into an agent? The answer lies in what happens between the user's input and the system's output. While a simple chatbot makes a single call to a language model and returns text, cognitive language agents engage in sophisticated internal processes before deciding what to do.
Researchers have developed a framework called Cognitive Architectures for Language Agents (CoALA) that organizes these internal processes into three fundamental components: memory systems, action spaces, and decision-making procedures. Think of it as an operating system for AI agents—a structured way to understand what they store, what they can do, and how they choose what to do next.
Three Kinds of Memory for Three Kinds of Knowledge
Just as humans don't rely solely on immediate thoughts but draw on different types of memory, language agents need multiple memory systems to function effectively.
Working memory holds whatever the agent is currently focused on—recent observations from the environment, current goals, and intermediate thoughts. When you see an agent reason through a problem step by step, that reasoning is happening in working memory. Unlike a simple language model that forgets everything after generating a response, agents with working memory can maintain context across multiple interactions.
Long-term memory comes in three varieties. Episodic memory stores past experiences—specific events the agent has encountered. If a robot tried to grasp a cup and failed because it was too hot, that specific episode gets stored. Semantic memory holds general knowledge about the world. The same robot might learn that "coffee cups are typically hot after brewing" as a general fact. Procedural memory contains the actual skills and rules the agent can execute—how to grasp an object, how to search a database, how to generate a plan.
What makes modern language agents remarkable is that they can write to their own memories.
Internal and External Actions: The Complete Repertoire
Language agents can do more than just output text. Their action space divides into two domains: internal actions that modify their own state, and external actions that affect the world.
Internal actions include reasoning (thinking through a problem step by step), retrieval (pulling relevant memories into focus), and learning (writing new information to long-term memory). When an agent pauses before answering to consider multiple possibilities, it's engaging in internal reasoning. When it searches its memory for similar past situations, it's performing retrieval.
External actions ground the agent in reality. These might be physical actions like robot movements, digital actions like API calls or web navigation, or communicative actions like dialogue with humans or other agents. A customer service agent that can actually update your account information (rather than just telling you how to do it) has taken an external action.
The Decision Cycle: How Agents Choose What to Do
Perhaps the most critical component is how agents decide which actions to take. CoALA structures this as a repeated cycle with three phases: proposal, evaluation, and selection.
During proposal, the agent generates possible actions. A robot in a kitchen might propose multiple options: "pick up the apple," "open the refrigerator," or "ask for clarification." During evaluation, it assesses each option. Which action is most likely to achieve the current goal? Which is safest? Which has worked in similar situations before? Finally, during selection, it chooses one action to execute, then observes the result and begins the cycle again.
This deliberative process distinguishes cognitive agents from simple reactive systems. Rather than immediately responding to every input, they can pause, consider alternatives, simulate potential outcomes, and make reasoned choices.
From Simple Chatbots to Learning Agents
To understand how these components work together, consider three real examples of language agents, each building on the capabilities of the last.
ReAct, developed in 2022, combined reasoning and acting in digital environments like Wikipedia. When asked a complex question, it could reason about what information it needed, search Wikipedia, observe the results, and reason again. However, it lacked long-term memory—each new query started fresh, with no learning from past interactions.
Voyager, designed for playing Minecraft, added procedural memory and learning. It maintained a library of code-based skills—like "craftStoneSword" or "combatZombie"—and could write new skills to this library when it discovered useful behaviors. Over time, it became increasingly capable at surviving and thriving in the game, and these skills transferred to new tasks it hadn't been trained on.
Generative Agents went further by adding both episodic and semantic memory. In a simulated town, these agents stored records of their daily experiences (episodic memory), reflected on those experiences to form general insights (semantic memory), and used both to guide future behavior. One agent might attend a party, remember who they met, form opinions about different residents, and adjust their future interactions accordingly.
Building Better Agents: A Practical Roadmap
The CoALA framework doesn't just help us understand existing agents—it provides a blueprint for building better ones. Several practical insights emerge from thinking about agents in this structured way.
Modular design matters. Just as software engineering benefits from reusable components, agent development should too. By separating memory systems, action spaces, and decision procedures, we can mix and matched components depending on the task. A customer service agent might need rich episodic memory of past interactions but limited physical actions, while a robot might need extensive procedural memory but minimal semantic knowledge.
Reasoning should be structured, not just prompted. Early approaches to making language models reason simply added "think step by step" to the prompt. CoALA suggests a more systematic approach: define clear reasoning procedures that update specific working memory variables. This makes agent behavior more predictable and easier to debug.
Learning means more than fine-tuning. When most people think of AI learning, they imagine updating model weights through additional training. CoALA expands this view to include multiple forms of learning: storing new experiences, reflecting on those experiences to form general knowledge, and even writing new code-based skills. These forms of learning are faster, more interpretable, and more flexible than traditional fine-tuning.
What's Next for Language Agents
Several exciting frontiers await language agent research. Perhaps most significantly, agents are just beginning to explore truly deliberative decision-making. Most current agents generate a single action and execute it. Future agents will more systematically propose multiple alternatives, simulate their likely outcomes, and choose the best path forward—much like humans mentally rehearse different approaches before acting.
The boundary between internal and external also raises fascinating questions. Is a Wikipedia database internal memory or external environment? The answer matters for how agents are designed and what safety measures are needed. Generally, if an agent can control and modify a resource, it's better thought of as internal memory; if the resource exists independently and may change unexpectedly, it's external.
Perhaps most intriguingly, the relationship between language models and agent architectures will continue to evolve.