fix(esh): revert the forced 10G — the link was flaking hard, operator called it

Both ends are back on autoneg at 1000 Mbps and stable; UDM SFP+2 restored to
autoneg True / setting_preference auto from the payload staged before the
original change. ~50 s of renegotiation, nothing rebooted.

The force failed progressively and I called it too early. Errors on the
ESH-Media side went 200 at link-up, 221 at 42 minutes, 416 by four hours, with
flapping the operator noticed before my soak had accumulated enough to raise
it. I read a two-minute flat window as a plateau and said so; a marginal link
declares itself over hours, not minutes.

Do not re-force this port. The cable is being replaced at the weekend and a
correctly-coded DAC negotiates 10G on its own.
This commit is contained in:
vh
2026-09-04 14:41:02 -07:00
parent 514ce7acc8
commit 054c098777
+23 -26
View File
@@ -3,30 +3,27 @@
2026-09-04. The link runs at 10G by an explicit rate override on the UDM side. 2026-09-04. The link runs at 10G by an explicit rate override on the UDM side.
**Read the pinned-port warning below before touching that port.** **Read the pinned-port warning below before touching that port.**
> ## ⚠ CURRENT STATE: forced 10G, KNOWN MARGINAL, held deliberately > ## CURRENT STATE: REVERTED to autoneg / 1G. The force is gone.
> >
> **Operator decision 2026-09-04: leave the override in place.** He has a > **2026-09-04 14:38 — the operator reported the 10G link "flaking hard" and I
> replacement cable and will fish it through the garage at the weekend. Do not > reverted it.** Both ends are back at 1000 Mbps, UDM SFP+2 is `autoneg True /
> revert it and do not re-diagnose it — this is a held position, not an > setting_preference auto`, ESH-Media is reachable and stable. The revert took
> oversight. > about 50 s of renegotiation and nothing rebooted.
> >
> The link is **not clean at 10G**: errors accumulate steadily (200 → 221 rx over > **The forced-10G experiment failed, and the failure was progressive.** Error
> 42 min, ~1 per 23 MB, roughly one bad frame in 15,000) and it dropped once, at > counts on the ESH-Media side over the life of the force:
> 10:12, recovering on its own. Nothing rebooted. TCP rides over it; expect
> retransmits and jitter rather than visible corruption.
> >
> When the cable is swapped: **remove the override** and let autoneg work. A > 10:03 200 link-up burst
> properly-coded DAC negotiates 10G on its own — that is the whole point. > 10:48 221 +21 in 42 min -- the first sign it was not plateauing
> 14:38 416 +195 over the next 4 h, plus user-visible flapping
link ESH-Media SFP+1 (port 11) <--DAC--> ESH-UDMPM SFP+2 (port 11, eth10) >
cable OEM "SFP-H10GB-CU3M", serial CSC200803000004, 3 m passive twinax > ⚠ **The lesson, and it is mine to own: I read a two-minute flat window as a
before both ends 1000 Mbps, autoneg on > plateau and reported it as one.** The counter simply had not moved yet. A
after both ends 10000 Mbps > marginal link declares itself over hours, not minutes — and the operator felt
> the flapping before the soak I left running had accumulated enough to call it.
This link backhauls the USW Flex XG "ESH-Office-Vuong", the E7-ESH-Media access >
point, and the Zigbee coordinator — the office switch and the AP both connect at > **Do not re-force this port.** The cable is being replaced at the weekend; a
10G and were squeezing through a 1G uplink, so this was a real bottleneck rather > correctly-coded DAC negotiates 10G on its own, which is the actual fix.
than a cosmetic one.
## The diagnosis, and what made it certain ## The diagnosis, and what made it certain
@@ -78,12 +75,12 @@ pre-change `port_overrides` array was written to a revert payload before the
forward one was sent. If a future attempt needs *both* ends forced, that is a forward one was sent. If a future attempt needs *both* ends forced, that is a
different risk class and needs the operator's sign-off, not an agent's. different risk class and needs the operator's sign-off, not an agent's.
## ⚠⚠ THAT PORT IS NOW PINNED — a 1G module will not link ## The port is NOT pinned any more — the override was removed 2026-09-04
UDM SFP+2 is `autoneg False, speed 10000`. Plug anything that is not a 10G While the force was in place, UDM SFP+2 was `autoneg False, speed 10000` and a
module into it and the link stays down with no obvious cause. If the DAC is ever non-10G module would not have linked there. That is no longer true: the override
replaced with a properly-coded one, **remove the override** and let autoneg do is gone and the port is back on autoneg. Recorded because the warning stood in
its job rather than leaving a pin nobody remembers. this runbook for four hours and someone may remember it.
## What the link bounce cost ## What the link bounce cost