You spend an evening building something with a model — a design, a plan, a whole little universe of decisions and constraints and names. It’s sharp, it’s context-rich, it gets it. The next morning you open a fresh chat and it sounds like a baby. It knows nothing about yesterday. Not because it got dumber overnight — because the thing you’d built lived entirely in a transcript that’s now gone.

That reset is the most honest demonstration I know of what these models actually are: a stateless brain riding on a stateful transcript. Every answer is computed only from the text visible right now. There is no “it” that persists between conversations. There’s just the scroll, and when the scroll ends, so does the mind.

Two different amnesias

It’s worth separating the two failures, because they have different fixes.

Within a conversation, the context window is fixed. As the transcript grows, the earliest exchanges get pushed out or attention-starved — the model over-weights what you said five minutes ago and forgets what you established on the first screen. And bigger windows don’t rescue you the way you’d hope: recall on a buried fact drops off sharply well before you fill even a very large window. A model that technically holds a million tokens will still lose a detail you dropped at token 100,000.

Across conversations, what feels like “memory” is a lossy background summary. It captures that you were working on something — “a clinical case,” “a migration” — but not the load-bearing specifics: the actual working hypothesis, the exact coordinate, the reason you rejected the obvious approach. And the summary gets denser over time, not more complete. It’s a compression, and compression throws things away.

Continuity is infrastructure, not a prompt

Here’s the reframe that turned this from a gripe into a project: humans don’t store the transcript. We store the map. You don’t remember the words of a conversation; you remember what it meant — a compressed graph of who and what and how-they-connect. The model has the words and not the map, which is exactly backwards from how you’d want to remember anything.

So giving a model continuity isn’t a matter of a cleverer prompt. It’s a layer of infrastructure that sits between the chat and the model: a structured, queryable store of what’s been established, automatic retrieval of the relevant parts into the context, and a path for new information to flow back into the store. That’s plumbing, not intelligence — storage, retrieval, and the unglamorous business of keeping a record correct over time.

Facts and stories don’t belong in the same store

The insight that actually surprised me — the one that reorganized how I think about this — is that facts and narrative need different storage and different retrieval, and almost every “chat with your documents” tool gets this wrong by using one mechanism for both.

That mechanism is vector similarity: embed everything, and retrieve by semantic closeness. It’s genuinely good for narrative — “what did we say that’s like this?” — where fuzzy, associative recall is the whole point. It is actively dangerous for facts.

Picture two variant identifiers that differ by a single character — c.5C>T and c.5C>A. To a vector index they are near-neighbors, because they are textually almost identical, and textual similarity is precisely what embeddings reward. But they are different facts, and retrieving one when you asked for the other is not a fuzzy-but-helpful answer — it’s a wrong one, in exactly the domains (clinical, financial, legal) where wrong is expensive. The property that makes semantic search useful for stories is the property that makes it lethal for facts.

So the substrate splits: an append-only, dated log for narrative, retrieved by meaning; and a structured store for hard facts, retrieved by exact key. Two shapes of memory, because there are two shapes of thing to remember.

The log records; the projection judges

The other load-bearing decision is that the raw observation log is the durable asset and everything else is disposable. Each record is append-only and stamped with the version of the extractor that wrote it. The graph, the embeddings, the aggregates — all of that is a derived view over the log, and when your extraction logic gets smarter, you re-run it across the whole history and lose nothing. It’s the same idempotent-replay discipline I’d trust in any pipeline: keep the raw, derive the rest, so you’re never one design change away from having thrown away the truth.

That separation — the log records; the projection judges — also keeps you honest about belief. History is what was observed; what you currently believe about it is a computed layer on top, and it should be allowed to change without rewriting the past. How you weight and reconcile those beliefs is its own long argument — I’ve made it elsewhere — but it only works if the layer underneath is an immutable record and not a running guess.

The hard part is extraction

Storing this stuff is easy. The hard part — the part that decides whether any of it works — is extraction: turning a messy conversation into clean, structured entries. Fully automate it and the model hallucinates, conflates two entities into one, and fails silently — you find out the store is corrupt only after you’ve trusted a bad fact. Do it by hand and it doesn’t scale past a toy. The only thing I’ve seen actually work is the boring middle: the model proposes an update and a human accepts it. Machine drafts the memory; person signs off. Anything else rots.

I’ll be clear-eyed about where this left me: I never shipped the heavyweight version. It stayed a design, and the reasons are the useful part. The elaborate machinery only earns its keep in the messy, contested, must-be-auditable case; for ordinary “remember what we talked about” the shipping products make simpler compromises and do fine. And native memory in the chat tools is improving fast enough that a big abstraction layer could be obsolete inside a year. So the move that survived contact with all that doubt was the smallest one: two plain-text files — a facts.md and a log.md, thirty lines, ten minutes, no code. The elaborate version is still a design doc. The thirty-line one has been running my real conversations for months, and I’ve stopped feeling like it owes me anything fancier.

Sources & further reading

• Liu et al. — Lost in the Middle: How Language Models Use Long Contexts — recall drops off well before the window is full.
• Vaswani et al. — Attention Is All You Need — the transformer underneath the “stateless brain.”