Virtual context (LLM)
Virtual context is the agent-memory pattern introduced by MemGPT (now Letta) — a small in-context working memory plus an external archival memory the model can read from and write to as the conversation grows.
MemGPT (Packer et al., 2023) introduced virtual context inspired by OS virtual memory. The model sees a small main context (a few thousand tokens of recent conversation + system prompt + scratchpad) and has explicit functions to read from / write to an archival memory layer. As conversation grows, the model selects what to evict to archival and what to recall. The pattern lets the model handle conversations longer than the underlying model's context window. Letta (the productised version) and several derivatives ship this architecture in 2026.
When to use virtual context (llm)
- Long-running assistants without persistent fine-tuning.
- Agent memory experiments and research.
Common mistakes
- Treating virtual context as a panacea — it adds latency + complexity vs simpler memory layers.
- Skipping eviction policy thought — what to forget matters as much as what to keep.
FAQ
What is virtual context (llm)?
Virtual context is the agent-memory pattern introduced by MemGPT (now Letta) — a small in-context working memory plus an external archival memory the model can read from and write to as the conversation grows.
When should I use virtual context (llm)?
Long-running assistants without persistent fine-tuning. Agent memory experiments and research.
What are the most common mistakes with virtual context (llm)?
Treating virtual context as a panacea — it adds latency + complexity vs simpler memory layers. Skipping eviction policy thought — what to forget matters as much as what to keep.
Related terms
- Context distillation — Context distillation summarises an agent's growing conversation history into a compact representation, so each step's input stays small while preserving the relevant signal.
- Context window — The context window is the maximum number of tokens — system prompt, conversation history, retrieved documents, and the response — that a language model can process in a single turn.
- Stateful agent — A stateful agent persists state — memory, learned facts, long-running context — across sessions, in contrast to stateless agents that start fresh on every conversation.
Sources
Last updated: 2026-06-01. Raw markdown: https://promtable.com/glossary/virtual-context.md.