Spark
Posts tagged “Spark”.
- How Much of a Spark Plan Actually Runs on the GPU?
One late CPU island barely hurt a Spark RAPIDS plan; two separated islands made the hybrid execution 33% slower than CPU.
- The Cluster Was Idle the Whole Time
A Fabric notebook spent nearly three hours copying 50,000 files serially from its driver while the Spark executors remained idle.
- The Missing Column That Didn't Crash
Spark mapped a shortened CSV to an explicit schema by position, so the job stayed green while values silently shifted into the wrong columns.