In 2015 I put a game’s collision detector on a GPU expecting the payoff everyone promises: a thousand cores, a big speedup. I got a wash. The reason has stuck with me longer than almost anything else I learned in either of my degrees, because it turned out not to be about that project at all.

The GPU wasn’t slow. The bus was. My data lived in main memory, and getting it across PCIe to the device cost more than the parallel cores saved. The arithmetic was free; the travel was the whole bill. That was the first time I really felt a thing I’ve now watched prove itself in four different domains: compute is cheap, and moving data is expensive, and the interesting engineering is almost always about the second one.

The same lesson, four times

Once you’ve felt it, you see it everywhere.

In the engine work, this was practically the curriculum. Before any of it touched a GPU, the CPU side drilled in that layout is performance. SIMD works best on aligned data, so to use it well you arrange memory around the hardware instead of your convenience. You split the “hot” fields an inner loop touches every frame from the “cold” ones it rarely does, so a cache line carries only what the loop needs. You stop trusting the general-purpose allocator and build pool allocators that control where things actually live. You preprocess assets offline into contiguous, load-in-place buffers so that at runtime a whole object is one sequential read with no pointers to chase. None of that makes the arithmetic faster. All of it makes the data cheaper to move.

Then the animation pipeline went onto the GPU, and it was the same lesson at higher stakes. The keyframes, bind poses, bone hierarchy, and weights that the CPU version had chewed through got relocated into GPU buffers, and the interpolation, the skinning-matrix math, and the vertex deformation all happened in shaders — each stage writing its output into a buffer the next stage read, so the data stayed on the GPU instead of round-tripping back to the CPU. I even wrote a step that compressed the keyframes before they got there, dropping any frame you could predict from a neighbor and cutting some animations by 85–90%. Faster? A little. But mostly it was less data to move. Parallel hardware is a machine for doing an enormous amount of work — if you can feed it, and feeding it is fundamentally about moving data, not crunching graphics.

In the search engine I built by hand, my first index was a dense document-term matrix that dutifully stored millions of zeros and fell over on a modest corpus. The fix — an inverted index of just the postings that exist — didn’t change the cosine math one bit. It changed how much data had to move to answer a query: from all of it to only the documents that contain a query term. Same computation, a fraction of the motion.

In big-data work, raw distributed processing makes the point impossible to miss: the shuffle between stages is the only coordination you get, and the entire craft is arranging things so the large dataset never has to move across the network. Broadcast the tiny lookup table; keep the huge one still; filter before you shuffle. Every optimization is a way to not move bytes.

In my deep-learning capstone, the model was a library call. The project was that a few thousand 3D volumes wouldn’t fit in memory, so the real work was a preprocess-to-disk pipeline and a custom loader streaming batches in. Nobody writes a paper about that part. It’s where all the time went.

And now, in data engineering, moving data around is the visible job — but the hard-won part is moving less of it. Incremental loads instead of full reloads. Watermarks so you never re-read what you’ve already seen. The whole reason I keep saying a pipeline is a shape of data, not a shape of code is that the code is trivial and the data motion is where the cost and the bugs live.

Four domains, one shape: the naive version computes fine and drowns in data movement, and the good version rearranges the data so less of it has to travel. The primitive that wins is the one that moves the least. Moore’s law made compute cheap; it did nothing for the speed of light or the width of a bus.

The compute box is small and cheap in every row. The data-movement arrow is the bill.

Following the hardware, not switching fields

There’s a personal thread in this that I only noticed in hindsight.

I was doing GPU data-parallelism in games in roughly 2013 to 2015 — pushing geometry and animation onto parallel hardware, learning to think about memory layout and transfer cost. That happens to be the exact window when the same hardware was quietly becoming the substrate of the deep-learning explosion. The GPU stopped being a graphics card and became the thing you trained neural networks on, and the whole field pivoted onto it.

So when I moved from game development into data science, it never felt like switching fields. The thing I’d been playing with — get data onto parallel hardware, lay it out so the cores stay fed, pay attention to where it lives — was the thing about to eat machine learning. I wasn’t changing tracks. I was following the hardware, and the hardware led from collision detection to convolutional networks because underneath both of them was the same question: how do you keep thousands of cores busy without spending all your time moving data to them?

My interests look, from outside, like a wandering path through games and statistics and deep learning. From inside it’s one continuous investigation of a single stubborn fact, chased across whatever domain was making it most vivid at the time.

The instinct I actually kept

I don’t get excited about faster compute anymore. When someone shows me a system, the first thing I want to know is where the data has to travel and how often — because that, far more than the cleverness of the computation, is what decides whether the thing is fast, cheap, and able to survive contact with real scale.

It’s the oldest instinct I own as an engineer, and it started with a collision detector that spent all its time idling — starved, waiting on a bus it had already saturated. Every project since has been a larger version of that same starved collision detector.