Years ago, in grad school, I wrote the networking for a small multiplayer game. Two players, objects moving around, collisions, the usual. At the time I thought of it as a game-programming problem. It took me a while to notice I’d actually built a small distributed system, and that every hard part I hit was a distributed-systems problem in a game costume.

The shape it settled into was three moves, and I’ve been making those same three moves in data infrastructure ever since:

• Optimism at the edge — predict locally so the player never feels the network.
• Authority at the core — one server owns the truth so everyone agrees.
• A reconciliation loop — for when the optimistic edge and the authoritative core disagree, which they always eventually do.

Optimism at the edge is a lie you have to reconcile

If you wait for the server to confirm every move, the game feels like moving furniture through syrup. So you predict: the client shows the result immediately and hopes the server agrees.

My first attempt at this was naive in an instructive way. I took the object’s last velocity and added it to its current position — move it forward as if nothing had changed. In the demo, on my machine, it looked like it was working. It was only later that I realized it wasn’t the right solution at all: I was ignoring latency entirely. The correct version had to reconstruct the missing frames — how much time had actually elapsed, including the packet’s travel time — and advance the object by that, not by one blind step.

This is the whole story of an optimistic write. It looks fine in the demo because the demo has no latency. Under real network delay, the naive version is quietly wrong, and you only find out when you stop testing against yourself. The demo is not the system. The gap between “looked like it was working” and “was actually correct” was exactly the round-trip time I’d pretended didn’t exist.

Authority at the core, on purpose

The counterweight to all that optimism is a rule I wrote down for myself at the time: do the final collisions and state changes in one atomic location — the server.

Concretely, when something important happened on a client, I didn’t apply it locally and hope. I sent the event across the network into the host’s own input queue and let the host decide. It felt slower and more roundabout, and it was — but it’s what kept the two players’ worlds from drifting into two different games. One authority, one place where state actually changes, everyone else downstream of it.

I didn’t have the vocabulary for it then, but this is the single-writer principle. It’s the same instinct I now reach for in data systems: RLS and read replicas everywhere you like, but one path is allowed to commit the truth. Consistency is much easier when only one component is permitted to be right.

Reconcile by converging, not teleporting

Prediction is going to be wrong sometimes. The server says the object is here; the client, running ahead optimistically, put it there. Now what?

The naive fix is to snap it to the server’s position. It’s correct and it looks terrible — objects teleport every time a correction arrives. What I did instead was interpolate: ease the object from where the client guessed toward where the server says, with a slow-decaying blend, so it converges over a few frames instead of jumping.

That’s reconciliation, and “converge, don’t teleport” turns out to be the general lesson. When two replicas disagree, a hard overwrite is correct and jarring; a gradual reconciliation toward the authority is usually what you actually want. The truth wins either way — the question is whether the seam shows.

The bug that taught me the most: what state do you even send?

The single hardest thing in the whole project sounds trivial written down: getting velocity to replicate correctly. I tried linear velocity — wrong. Normalized — wrong. Angular — wrong. Nothing behaved until I sent per-frame velocity: the change computed over a number of frames rather than over game time.

The lesson stuck because it generalizes way past games. It is not enough to replicate “the state.” You have to replicate the right representation of the state — the invariant that stays meaningful on the other side of the wire, under a different clock and a different frame rate. Send the wrong unit and everything downstream corrupts silently, in ways that look almost right, which is the worst way for something to be wrong. Picking what to put on the wire is a real design decision, not a serialization detail.

The part that surprised me: I’d reinvented an ordered log

Here’s the bit I didn’t expect to still be thinking about years later.

To make any of this debuggable, I’d built the whole thing around a data-driven queue: every action — input, movement, collision, spawn — was a message on an input/output queue. No object reached into another object’s guts; everything went through the queue. And that one decision bought a pile of properties I didn’t fully appreciate at the time. Full observability, because nothing happened off the record. Record and replay, because the queue was the history. The ability to develop offline on a single machine by feeding the same message stream. Even reproducible randomness, by having the server seed it and broadcast the result as just another message.

I had, without the words for it, stumbled into event sourcing: an ordered log of messages as the source of truth, with state as a fold over that log. It’s the same idea I now take seriously in data infrastructure — that a durable, ordered sequence of events you can replay is worth more than the current-state snapshot, because the log can rebuild the snapshot and the snapshot can’t rebuild the log.

One more practical note from that queue: not every message wanted the same delivery guarantee. Some had to arrive and arrive in order; some were fine to drop; some just needed to arrive eventually. Forcing everything through a single reliable-ordered channel — TCP’s default bargain, timeliness sacrificed for order — is the wrong trade for a game, and often the wrong trade for a system. Choose the guarantee per stream, don’t pay the strictest tax on all of it.

The same three moves, ever since

Optimism at the edge. Authority at the core. Reconciliation between them, over an ordered log of events. I keep rebuilding that exact structure in data systems — optimistic UI over an authoritative API, a single writer for the truth, a reconciliation job for when they drift, an event log underneath it all.

Games just taught it to me first, under a harsher tutor. In data infrastructure the network mostly behaves and the failures are occasional. In netcode, latency and loss are the environment — present in every single frame — so you learn the lessons faster, because nothing ever lets you pretend the network isn’t there.