Layered Agent Memory

The memory architecture of a personal always-on agent, arranged in three tiers with different freshness, cost, and recall characteristics. Tier one is a handful of markdown files appended to every context right after the system prompt: in Hermes, soul.md (the agent's personality — effectively the system prompt; ships empty and falls back to a stock "you are a virtual assistant" prompt until written), memory/user.md (facts about the operator, auto-updated whenever the agent learns something about you), and memory/memory.md (arbitrary useful facts — tool usage, workflows, learnings). Tier two is the internal transcript store: every session's full transcript in SQLite, including a bare-text table that makes similarity search over past conversations easy; this is also where the Agent Gateway pulls message history from. Tier three is optional external memory providers (mem0, SuperMemory, Honcho, …) — specialized services with their own recall mechanisms, off by default, which when enabled inject a past-sessions summary block into context. The tiers are kept honest by the loop's memory-update tail: after every response the agent checks whether the exchange held anything worth writing down — which is what makes the agent learn continuously from use.

Claims

Layer agent memory into always-in-context markdown, an internal transcript DB, and optional external providers — each tier trades context cost against recall depth. principle — durable: the always-loaded tier must stay tiny, the transcript tier is complete but needs search, the external tier adds recall intelligence at integration cost.
A memory-update stage at the tail of every turn — analyze the exchange, write what's worth remembering — is what makes an agent continuously learn from use. principle — the per-turn, in-loop form of the persist-memory rule on Agent Loop; without it every tier is write-never.
Separate the personality file from learned memory: soul.md is operator-authored identity; user.md/memory.md are agent-written learnings. best practice — context: personal agents; the split keeps who-the-agent-is stable while what-it-has-learned accumulates. The source recommends writing your own soul.md rather than living with the stock default prompt.
Store full session transcripts in SQLite, with a bare-text table for similarity search. best practice — context: recalling prior conversations without a heavier retrieval stack; the same SQLite-transcript substrate OpenClaw searches in How AI Agents Search Their Memory — Hybrid Retrieval, in Practice (OpenClaw), seen here from the architecture side.
Query external memory after the first response, not before — answer first, then recall related history to anticipate the follow-up. best practice — context: external providers add per-query latency and cost; deferring recall one turn keeps first answers fast, at the price of first-turn amnesia (the source's workaround: describe what you want remembered in message one, ask in message two). The analogy given: a human answers, then remembers related conversations while listening.
The source states external memory is not configured by default and most users never enable it, though the presenter recommends it — several supported providers are free and it markedly improves what the agent retains about you. observation — attributed; provider mechanisms differ (similarity search vs full-history-plus-LLM-extraction vs enriched LLM queries) and the source defers comparison to a dedicated video.
Canonical three-tier framing: internal knowledge (weights) / context window (short-term) / external sources like RAG (long-term); route always-needed information into training, rarely-needed information into long-term memory, and immediate context into the window. principle — the AI Engineering textbook arrives at the same internal/context/external tiering as this concept independently, and adds the placement rule (what belongs in which tier by access frequency). Two sources now agree on the structure.

AI Second Brain — tier one is another independently-built instance of "agent memory is just markdown files," here maintained by the agent itself.
Agent Loop — the memory-update tail is a first-class stage of the conversational loop, not an offline process.
Agent Gateway — reads the transcript tier to rebuild per-channel conversations.
Context Compression — the complementary working-memory mechanism: compression lossily shrinks in-window history; the memory tiers are where durable facts must land first.
Self-Improving System — per-turn memory update is the smallest-grain compounding mechanism: automatic and in-loop, versus the periodic human-driven improvement loop.
Hybrid Retrieval — what the read path over the transcript tier grows into as recall needs mature.
Extended Mind — the principle altitude above the internal-vs-external tiering: on the parity view the boundary the tiers are named by is not a cognitive boundary at all; every tier is part of one extended system (Cognitive Scaffolding).
Distillate: Hermes Architecture EXPLAINED: Memory, Context & Gateways
Distillate: Andy Clark — What is Extended Mind? (Closer To Truth)
Distillate: AI Engineering in 76 Minutes — Chip Huyen's Book, Speedrun — independent textbook corroboration of the internal/context/external three-tier memory structure.

Layered Agent Memory

Claims

Related

Linked from