fix(gateway-chat): guard max_tokens against NaN from an empty field
An empty or non-numeric Max tokens field makes parseInt return NaN, and
JSON.stringify serialises NaN as null. The server reads null as 'no
max_tokens supplied' and substitutes its own default -- which is
indistinguishable from the UI ignoring the field, and is the most likely
explanation for a typed value appearing to have no effect. Falls back to
the same 4096 the input defaults to.
Ruled out on the way to this, all measured rather than assumed:
- LiteLLM caps nothing: max_tokens=None on both aliases, no max-token
keys in litellm_settings or general_settings.
- The gateway honours large values end-to-end: 5,346 completion tokens
returned at max_tokens=8192, finish=stop.
- The UI has ONE chat send path, no duplicate element ids, a standard
getElementById helper, and the request body is never mutated after
construction -- so the field is read live at send time.
Remaining client-side cause if it recurs is a stale cached page: nginx
serves this file with only Last-Modified/ETag and no Cache-Control, so an
already-open tab will not re-fetch. ETag changes on each deploy, so a
reload picks it up.
This commit is contained in:
@@ -249,7 +249,12 @@ async function send(){
|
||||
model: $('model').value.trim() || 'gen',
|
||||
messages,
|
||||
temperature: parseFloat($('temp').value),
|
||||
max_tokens: parseInt($('max').value, 10),
|
||||
// Guarded: an empty or non-numeric field makes parseInt return NaN, and
|
||||
// JSON.stringify serialises NaN as `null` -- which the server reads as
|
||||
// "no max_tokens supplied" and silently substitutes its own default.
|
||||
// That is indistinguishable from the field being ignored. Fall back to the
|
||||
// same 4096 the input defaults to.
|
||||
max_tokens: parseInt($('max').value, 10) || 4096,
|
||||
stream: true
|
||||
// intentionally NO `tools` — vLLM 400s on an empty array.
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user