Idempotency
Posts tagged “Idempotency”.
- The Duplicate Files That Weren't Duplicates
File checksums catch identical deliveries, but overlapping business rows need set and multiplicity comparisons that answer a different question.
- A Data Pipeline Is a State Machine Wearing a DAG
Microsoft Fabric pipeline runs can succeed or fail independently of durable data state. Model retries, publication, and reversal as explicit transitions.
- I Tried to Make Structured File Intake Boring
I tested one configuration-driven intake path against clinical-trial, provider-registry, and public-transit files, including bad rows and retries.
- Checksum, Not Filename
Filenames cannot distinguish corrected files from reruns. Content fingerprints give ingestion pipelines a stable identity for duplicate detection.