For my grad-school capstone, a partner and I set out to classify autism from brain MRIs with a 3D convolutional neural network. We started where everyone starts — from a tutorial. A clean, well-known Keras example that trains a 3D CNN on CT scans in about two hundred lines: it downloads a tidy, pre-split dataset, builds the model, trains, and prints a number. Start to finish, it runs in a few minutes.

Our final model landed at 61% accuracy. Getting from that tutorial to that hard-won 61% took months, three separate preprocessing pipelines, and around 25 hours of GPU time — and it taught me more than a 95% would have, because everything I actually learned lived in the gap between the tutorial and the data.

(Credit up front: this was a two-person capstone, the work was shared, and the starting notebook was a public Keras example, not ours. What follows is what I took from it.)

The one-line data download becomes an ETL job

The tutorial’s data step is a single call: download a zip, already split into train and validation, volumes already uniform. That one line is the entire dataset problem, solved for you, invisibly.

Real data does not arrive like that. Ours came from a public research archive — an open bucket with per-site folders, paginated listings, inconsistent naming. Getting it meant pulling site by site with pattern-matching over the listings, reorganizing and renaming files into something uniform, and then — the part no tutorial mentions — recovering the labels. The labels lived in separate phenotype tables, one per collection site, with mismatched column names and even different text encodings, and you had to join scans to labels across all of that before you had anything a model could learn from.

None of this is modeling. All of it is required. The demo hands you a clean zip; the real project hands you a scavenger hunt across a dozen sites, and you’re an ETL engineer for a week before you’re ever a machine-learning one.

”Resize the image” is a modeling decision in disguise

In the tutorial, every scan is the same size, so resizing is a throwaway line. In real data every scan was a different shape, and “what size do I make them” became a decision I had to actually study — I ended up histogramming the distribution of volume dimensions across the whole dataset just to choose a target, because there was no obvious right answer.

We didn’t even settle on one. We shipped three representations and compared them: a 128×128×128 cube, a differently-cropped version, and — reaching for a completely different angle — a “video” of a handful of 2D slices pulled from each volume. The 2D-versus-3D-versus-slice-stack tradeoff that papers discuss in the abstract was, for us, three preprocessing pipelines we built and ran because we genuinely couldn’t tell in advance which framing the signal would survive.

You can’t hold it in RAM, so boring infrastructure wins

A few thousand 3D volumes is small as a number of examples and enormous as bytes. It does not fit in memory. The tutorial never has to care; ours did, from day one.

The fix was deeply unglamorous and completely necessary: preprocess everything once, write the processed volumes to disk as arrays, then feed training from disk with a custom data generator that loads a handful of volumes at a time. The stock image-loading helpers couldn’t handle our file types, so we wrote our own loader. There’s no cleverness in any of it. It’s the plumbing that decides whether the interesting part can run at all — exactly the kind of boring infrastructure that never shows up in a demo and always shows up in the real thing.

Colab versus your own iron

A fork runs through the whole project: the same notebooks exist twice, once for Google Colab (mount Drive, stream the data in) and once for a local workstation with a real GPU. The workstation trained at roughly fifteen minutes an epoch and something like 25 hours total across all the experiments. That’s the actual, unhidden cost of “just train a 3D CNN a few different ways” — a number the tutorial’s two-minute run will never prepare you for.

The honest 61%

Here’s where it landed: about 64% on validation, 61% on test, against a majority-class baseline of roughly 52%. Published models on richer brain-imaging data reach 79–95%. Ours misclassified the disorder group more often than the control group. By the standards of a demo, that’s a disappointing result.

By the standards of the truth, it was the most useful part of the project. Structural MRI alone is a genuinely hard signal for this task; the data is pooled from many sites with different scanners, which injects variation the model can’t separate from the thing you actually care about; and a few thousand examples is very little to teach a 3D CNN a subtle pattern. A small, honest gain over baseline — with every caveat stated plainly — is worth more than an impressive number I couldn’t defend. If we’d gotten 95%, my first job would have been to distrust it and go hunting for the leak.

That instinct is the thing I kept: the number that looks best is the one to check hardest, and a result you can trust beats a result that merely impresses.

The 5%

The tutorial was maybe 5% of the work, and it was the only part that felt like machine learning. The other 95% — scavenging the data, recovering labels, choosing a shape, engineering around memory, paying for the compute, and being honest about a mediocre result — was the actual project. None of it is in the demo. All of it is the system.

That’s what I took from a capstone that “only” reached 61%: the demo is not the system, and on real data the demo is barely even the beginning.