docs(ops-lessons): filter on the artifact, not the name pointing at it — measured 4.7x inflation
Quantifying exposure to a gate-failed tune: counting by the gateway alias gave 363 rows, counting by the artifact gave 77, because the alias had carried three different models that day. Wrong in the direction that looks careful.
This commit is contained in:
@@ -359,6 +359,28 @@ themselves: they waste attention and misdirect, but they cannot silently destroy
|
||||
a good artifact. A post-mortem that lumps all three together loses the half that
|
||||
decides what to do about it.
|
||||
|
||||
### Filter on the ARTIFACT, not on the name pointing at it
|
||||
|
||||
Measured 2026-09-09 while quantifying how much traffic had reached a tune that
|
||||
failed a safety gate. The gateway alias `trial` had pointed at three different
|
||||
artifacts across the day, so the obvious query — *"how many calls to `trial`?"* —
|
||||
answers a question about a **name**, not about the **thing**:
|
||||
|
||||
| filtered on | rows |
|
||||
|---|---|
|
||||
| `model_group = 'trial'` (the alias) | **363** |
|
||||
| `model = 'hosted_vllm/erp-tune-v7-nvfp4a16'` (the artifact) | **77** |
|
||||
|
||||
**Wrong by 4.7x, in the direction that looks careful.** Reporting 363 would have
|
||||
overstated the operator's own exposure nearly fivefold, and nothing about the
|
||||
query would have looked sloppy — an alias is what a caller types, so counting it
|
||||
feels like counting usage.
|
||||
|
||||
**The rule:** when the question is *"what did this artifact do"*, filter on the
|
||||
artifact's identity, never on a mutable pointer to it. A name that has been
|
||||
repointed carries the history of everything it ever pointed at. Same family as §3
|
||||
(identity, not resemblance): an alias resembles the thing and is not it.
|
||||
|
||||
### A fourth variant: the instrument read a surface MID-TRANSITION
|
||||
|
||||
Added 2026-09-09 from a near-miss brokkr-smithy-dev caught and did not send.
|
||||
|
||||
Reference in New Issue
Block a user