"An agent with no memory is like a brilliant colleague with amnesia — helpful for a moment, but you have to reintroduce yourself every time."
Learning Objectives
By the end of this module, you will be able to:
- Tell short-term context from long-term memory
- Understand how agents remember user preferences
- Recognize the risks of bad memory
- Respect privacy when storing memory
1. Two Kinds of Memory
Agents remember in two very different ways:
| Short-term context | Long-term memory | |
|---|---|---|
| What it is | The current conversation, held in the context window | Saved facts that persist across sessions |
| Lasts | Until the window fills or the chat ends | Days, months — until deleted |
| Analogy | What's on your desk right now | Your filing cabinet |
| Example | "The file I just pasted" | "This user always wants replies in Spanish" |
The key difference is where the information lives. Short-term context is everything currently inside the model's window — it vanishes when the conversation ends. Long-term memory is written to an outside store (a database or file) so it survives, and the agent pulls the relevant bits back into context when they're needed.
Key idea: Short-term context is working memory for the task at hand. Long-term memory is a store the agent writes important things into and looks them up later. Good agents use both — and know what belongs in each.
Explain like I'm new: Your desk holds only what you're using right this minute, and it gets cleared each night. Your filing cabinet holds things worth keeping, and you walk over to grab a folder only when you need it. An agent works the same way: desk = context, cabinet = long-term memory.
2. User Preferences
The most valuable long-term memory is usually user preferences — your name, tone, recurring choices ("book aisle seats," "summarize in bullet points"). Remembering these is what makes an assistant feel personal instead of generic.
Real-world use case: The first time you tell a travel agent "I'm vegetarian and I hate red-eye flights," it saves that to long-term memory. Months later, without you repeating a word, it books daytime flights and requests vegetarian meals. That small act of remembering is the difference between a tool you tolerate and one that feels like it knows you.
3. The Risks of Bad Memory
Memory cuts both ways:
- Remembering wrong things — a mistaken "fact" saved to memory gets repeated forever.
- Remembering too much — cramming everything into context confuses the model and costs more.
- Stale memory — an old preference that's no longer true ("still" booking a hotel in a city you moved away from).
Common mistake: Dumping the entire history into the context window every turn. More text isn't more intelligence — it dilutes focus, slows responses, and raises cost. Keep only what's relevant to the current step.
Because the context window has a fixed size, treating it like an endless bucket also has a hard ceiling: eventually the oldest, sometimes most important, information falls out. Being selective isn't just cheaper — it keeps the agent focused on what actually matters right now.
So how does an agent decide what is worth saving to long-term memory? A common approach is to save the things a person would jot in a notebook: durable facts about the user ("prefers aisle seats"), stable preferences ("always reply formally"), and important decisions ("chose the annual plan"). What it should not save is the throwaway detail of a single task — the exact wording of one question, or a file that's only relevant this once. A useful test: "Would this still matter next week?" If yes, it may belong in long-term memory. If it only matters for the current task, let it live in short-term context and disappear when the task ends. Getting this line right is most of what separates memory that helps from memory that clutters.
4. Privacy Matters
Memory means storing data about people, so privacy is essential:
Beginner tip: Store the least you need, tell users what you remember, and make it easy to view or delete. Never save sensitive data (passwords, IDs, health details) into agent memory unless you truly must — and then protect it carefully.
Key Takeaway: Agents use short-term context (working memory for the current task) and long-term memory (a store that persists, ideal for user preferences). Bad memory — wrong, stale, or bloated — hurts more than it helps, so keep it relevant. And because memory stores personal data, handle it with real privacy care.