esh-pfi-infrastructure/persistent-memory.d/2026-07-31-kimi-k3-reasoning-cap.md

[2026-07-31] kimi-k3 "output cap" root-caused = a ~16384 REASONING-token ceiling, not an output cap; fix relayed to heid, NOT applied gateway-side.

heid reported that kimi-k3 (the primary route = Kimi Code coding endpoint openai/k3 @ api.kimi.com/coding/v1) silently degraded its cross-frontier panel: on large/reasoning-heavy dispatches, completion_tokens: 16381 exactly, content empty, reasoning_content ~64KB, finish_reason: **stop** (a truncation mislabeled as a clean stop). max_tokens: 100000 in the request was not honored.

Investigation arc (a clean cross-frontier-triage + verify-on-the-wire case):

1. My first read: a flat ~16384 OUTPUT cap; fix = a LiteLLM stop→length relabel callback (heid's fallback ask). Confirmed the cap isn't in our LiteLLM config (no max_tokens clamp on the route).
2. Operator routed a fix-research pass to dvalin-smithy-dev + bil-smithy-dev (independent). Both CONVERGED (docs-based): max_tokens is a deprecated alias on Kimi/Moonshot; the canonical field is max_completion_tokens (default 131072, max 1M); the coding endpoint defaults output to 16384; fix = send max_completion_tokens + reasoning_effort via extra_body (drop_params-safe).
3. heid's live data REFUTED the docs hypothesis: a later dispatch hit completion_tokens: 18455 (ABOVE 16384) cleanly, with reasoning_tokens: 16198 (just under 16384) and content present. So COMPLETION is uncapped; the bound is on REASONING at ~16384. When a hard task's thinking exhausts that budget, nothing's left for content → empty answer under stop.
4. I proved it on the wire — ran heid's real 500KB failing bundle direct at both endpoints (bypassing LiteLLM so reasoning_effort isn't dropped): default effort → 504/timeout (the failure); reasoning_effort: low → reasoning ~12–13.5k (under the ceiling), content returns (6–7.6k chars), on BOTH coding AND general endpoints. So re-routing to the general endpoint buys nothing — the fix is the effort param, and it works on the wire.

THE FIX (caller-side, no shared-gateway change/restart): send reasoning_effort via extra_body on kimi-k3 dispatches (low for large bundles). LiteLLM drop_params: true strips the top-level reasoning_effort — which is exactly why heid's earlier reasoning_effort: low was a no-op. extra_body survives drop_params (the house GLM-thinking pattern). Tradeoff: low effort = shallower reasoning, but a complete answer beats today's empty one.

Relayed to heid to validate on a real round (the one unconfirmed hop is whether extra_body survives OUR LiteLLM). Backstop if it doesn't: add allowed_openai_params: ["reasoning_effort"] to the kimi-k3 route in the gateway config — that IS a shared-gateway change + a ~10s restart (blips all consumers), so it needs a heads-up.

Gateway = LiteLLM on ana-docker 10.250.50.70:4000; kimi-k3 config in stacks/litellm/conf/config.yaml (see Recent-decisions [2026-07-25] Kimi K3 wiring). No gateway change was made this session. Failing dispatch on record: 01KYTASKTY3T (jackdaw-dev bug-hunt).