Performance
Posts tagged “Performance”.
- The Row Count Was the Wrong Way to Choose Between SQLite and JSONL
At 10,000 records SQLite already beat unindexed JSONL on lookup, aggregate and update, while append stayed close. The operations decided it, not the row count.
- DuckLake on One Machine
I put a small DuckLake, DuckDB, and marimo data platform through one complete job, from 200,000 to 100 million events.
- Five 4K60 Streams Were Fine. The Sixth Was Not.
A paced NVDEC-CUDA-NVENC pipeline found the live-video capacity boundary and showed why a NumPy round trip cost more than the CUDA work.
- The Radio Won't Wait for Your FFT
A paced I/Q replay found where an eight-core SciPy pipeline began missing deadlines and dropping blocks, and how far an RTX GPU moved that boundary.
- I Wrote the Same GPU Operation Six Ways
Six implementations of one row-scoring operation show why a 324× resident GPU kernel becomes 10.4× once host-device transfers are included.
- I Moved an Entire ML Pipeline to the GPU
A complete GPU ML pipeline reached 4.74× at one million rows, while transfer costs and tiny inference batches often kept the advantage on CPU.
- 100 Million Points on One GPU
GPU spatial joins crossed by one million points, a rectangle barely won at 100 million, and one quadtree setting silently lost matches.
- How Many Queries Pay for a Vector Index?
CAGRA searched vectors up to 76× faster than exact cuVS, but its index build required roughly 45,000 queries to amortize.
- How Much of a Spark Plan Actually Runs on the GPU?
One late CPU island barely hurt a Spark RAPIDS plan; two separated islands made the hybrid execution 33% slower than CPU.
- DuckDB, Polars, and cuDF on One Analytical Pipeline
Across 100,000 to 200 million rows, GPU joins crossed early, strings never crossed, and partitioned Polars handled data larger than VRAM.
- When Does GPU Parquet Actually Pay Off?
Five Parquet readers show cuDF leading large scans, PyArrow keeping two small projected reads, and projection mattering more than engine choice.
- The Cluster Was Idle the Whole Time
A Fabric notebook spent nearly three hours copying 50,000 files serially from its driver while the Spark executors remained idle.