Files
esh-pfi-infrastructure/stacks
vh b6552e0546 fix(gateway-chat): raise max_tokens default 1024 -> 4096; document single-file bind-mount trap
Operator reported the gen seat 'cutting off'. It is not the seat. The chat
UI's max_tokens field defaults to 1024, and every thinking seat spends part
of that budget on CoT before emitting content, so the completion truncates
mid-sentence with finish_reason=length and reads as model degeneracy.

Measured through the gateway:
  gen            1024 -> finish=stop, 716w   (survives, but marginally)
  gen-reasoning  1024 -> finish=length, cut mid-word  <-- the symptom
  gen-reasoning  4096 -> finish=stop, 839w

Seat itself is clean: direct long-form generations return finish=stop with
complete sentences and a max repeated 6-gram of 1 (no degeneration), and
enable_thinking:false still holds on every non-thinking alias, so the
AEON swap did not cause this.

Also documents a trap that made the fix look like it had not applied:
compose bind-mounts a single FILE, and a single-file bind mount binds the
INODE. rsync writes-and-renames, producing a new inode, so the container
kept serving the old content while the host file showed the new value --
silently, with no error. docker restart does NOT clear it; the container
must be recreated. Verify against what the container sees, never the host
file. Applies to any file-source mount; directory mounts are unaffected.
2026-08-16 15:10:27 -07:00
..
2026-04-29 13:27:12 -07:00