For a grad-school independent study I built a collision-detection system on the CPU, got it working, and then ported it to the GPU with CUDA — expecting the satisfying speedup that’s supposed to be the whole point of moving work onto a thousand parallel cores.

The timer said it was a wash. At my scene size the GPU version ran in roughly one to four milliseconds; the CPU version ran in about one. No clear win, sometimes a loss, and my timing was too coarse to even prove which. By the standards of a demo, a disappointing result.

The failed speedup was more informative than the demo I had expected. It forced me to account for memory layout, transfers, divergent work, and the limits of my own timer. It was also the first time I understood that “GPU-accelerated” describes an implementation, not a benefit.

The tree had to become an array

On the CPU, my collision hierarchy was a tree of bounding volumes — a pointer-chasing structure, each node holding references to its children, traversed by following pointers. Perfectly normal, perfectly fast on a CPU that loves chasing pointers around a cache.

A GPU cannot work like that efficiently. Before a single thread could run, I had to flatten the tree into contiguous arrays: nodes became indexed records, and child pointers became integer offsets. The collision logic barely changed. Re-expressing the data so thousands of threads could reach it predictably was most of the port.

The port was really about memory layout, not the algorithm: the tree had to become an array before a single thread could run.

Whose hardware does the work — and where the data lives

Transfer cost decided the timing before kernel cleverness had much chance to matter.

My collision data lived in main memory, and the GPU reached it through a device proxy pointer — a way to let a kernel touch host memory directly. Convenient, and a trap: accessing main-memory data over that path was the slow lane, and the cost of shuffling data across PCIe between host and device swamped whatever the cores saved. The GPU spent its time waiting on the road, not doing work.

What rescued it — partially — was CUDA streams: overlapping the transfers with computation so the two happened at once instead of in sequence. That helped “a lot,” in my notes at the time, especially for the slow proxy-pointer accesses. But “streams made the transfers hurt less” is a very different sentence from “the GPU was faster.” The transfer was always the story. Where the data lived mattered more than how clever the kernel was.

Trees are the wrong shape for SIMT

There’s a second, subtler reason it didn’t fly. A GPU runs threads in lockstep groups — they’re happiest doing the same work on uniform, independent data. My traversal was level-by-level through a hierarchy where different nodes had different numbers of children and different amounts of work. So the threads finished at wildly different times and sat idle waiting for the slow ones.

I didn’t have the word for it then; the word is divergence, with load imbalance behind it. Even without the vocabulary, the behavior was visible: irregular tree traversal gave neighbouring threads different amounts of work, leaving some idle while others finished.

The measurement could not support a win

Add it up and the result makes sense: at my scene size, there wasn’t enough independent work to amortize the cost of getting the data onto the GPU in the first place. Parallel hardware needs a big enough pile of uniform work that the transfer overhead disappears into the savings. Below that threshold — and I was below it — the CPU just wins, because it never had to ship the data anywhere.

My timer resolution was milliseconds, and the numbers I was comparing were themselves only a few milliseconds. I could not cleanly prove a speedup, so I did not claim one. Calling the project “GPU-accelerated collision detection” would have been technically true while allowing the title to imply a performance result the measurements did not establish.

At this scene size, moving collision detection to CUDA bought almost nothing. The project still changed how I evaluate parallel work: first account for where the data lives, then ask whether enough uniform work remains to amortize moving it.

More on Applied experiments Five 4K60 Streams Were Fine. The Sixth Was Not. →