For a grad-school independent study I built a collision-detection system on the CPU, got it working, and then ported it to the GPU with CUDA — expecting the satisfying speedup that’s supposed to be the whole point of moving work onto a thousand parallel cores.

The timer said it was a wash. At my scene size the GPU version ran in roughly one to four milliseconds; the CPU version ran in about one. No clear win, sometimes a loss, and my timing was too coarse to even prove which. By the standards of a demo, a disappointing result.

It’s also the single most useful thing I learned in two quarters, because the reasons it was a wash are the reasons parallel compute is harder than “just add more cores” — and, quietly, the first time it sank in that “GPU-accelerated” is an engineering detail, not a reason anyone should care.

The algorithm was never the hard part. The memory layout was.

On the CPU, my collision hierarchy was a tree of bounding volumes — a pointer-chasing structure, each node holding references to its children, traversed by following pointers. Perfectly normal, perfectly fast on a CPU that loves chasing pointers around a cache.

A GPU cannot work like that. Before a single thread could run, I had to flatten the whole tree into flat, contiguous arrays — nodes as indexed records, “children” as integer offsets into the same array, everything laid out so thousands of threads could stride through it in parallel without chasing a pointer anywhere. The interesting realization: the data-parallel port was a data-layout problem, not an algorithm one. The collision logic barely changed. Re-expressing the data so the hardware could touch it in parallel was 90% of the work, and it’s the part no “port it to CUDA” tutorial dwells on.

The port was really about memory layout, not the algorithm: the tree had to become an array before a single thread could run.

Whose hardware does the work — and where the data lives

Here’s what actually decided the outcome, and it wasn’t kernel cleverness. It was the bus.

My collision data lived in main memory, and the GPU reached it through a device proxy pointer — a way to let a kernel touch host memory directly. Convenient, and a trap: accessing main-memory data over that path was the slow lane, and the cost of shuffling data across PCIe between host and device swamped whatever the cores saved. The GPU spent its time waiting on the road, not doing work.

What rescued it — partially — was CUDA streams: overlapping the transfers with computation so the two happened at once instead of in sequence. That helped “a lot,” in my notes at the time, especially for the slow proxy-pointer accesses. But “streams made the transfers hurt less” is a very different sentence from “the GPU was faster.” The transfer was always the story. Where the data lived mattered more than how clever the kernel was.

Trees are the wrong shape for SIMT

There’s a second, subtler reason it didn’t fly. A GPU runs threads in lockstep groups — they’re happiest doing the same work on uniform, independent data. My traversal was level-by-level through a hierarchy where different nodes had different numbers of children and different amounts of work. So the threads finished at wildly different times and sat idle waiting for the slow ones.

I didn’t have the word for it then; the word is divergence, and load imbalance behind it. But the shape of the lesson was clear even without the vocabulary: parallel hardware wants uniform, independent work, and a tree is neither. Irregular, hierarchical, data-dependent traversal is exactly the workload that fights the hardware you’re running it on.

When the GPU doesn’t win — and saying so

Add it up and the result makes sense: at my scene size, there wasn’t enough independent work to amortize the cost of getting the data onto the GPU in the first place. Parallel hardware needs a big enough pile of uniform work that the transfer overhead disappears into the savings. Below that threshold — and I was below it — the CPU just wins, because it never had to ship the data anywhere.

The last lesson is the one I’m proudest of, and it’s not technical. My timer resolution was milliseconds, and the numbers I was comparing were a small number of milliseconds. That’s noise. I couldn’t cleanly prove a speedup, so I didn’t claim one. It would have been easy to round the story up to “GPU-accelerated collision detection” and let the CUDA in the title do the implying. But a result you can’t measure honestly is not a result — the number that would look best is exactly the one to distrust.

“GPU-accelerated collision” was an engineering fact that bought me almost nothing — until I fixed where the data lived, and even then, at my scale, barely. Which is the whole point. Two questions ended up mattering more than the acceleration ever did: whose hardware does the work, and whether there’s enough work to be worth the trip.