Update: reference/nemotron-3-super-running-parameters.md — Added "Nemotron EOS Token Leak" troubleshooting section with problem description, fix (stop sequence), and updated example commands including --stop flag for llama-server deployment

reference/nemotron-3-super-running-parameters.md

@@ -2,28 +2,23 @@
 created: '2026-04-09T18:07:12+00:00'
 id: 2a98c58e-a3e3-429b-87b0-dd9c199a6e91
 keywords:
-- Nemotron 3 Super
+- nemotron
-- NVIDIA
+- unsloth
-- 120B-A12B
-- temperature
-- top_p
-- min_p
-- GGUF
-- quantization
-- UD-Q4_K_XL
-- NoPE
-- 1M context
 - llama.cpp
-- llama-server
+- nvidia-nemotron-3-super-120b-a12b-gguf
-- MoE
+- nvidia
-- 12B active
+- super
-- tool calling
+- ud-q4_k_xl
-- reasoning tokens
+- reasoning
-- NVFP4
+- min_p
-- FP8
+- nvidia-nemotron-3-super-120b-a12b
-- BF16
+- huggingface.co
+- running
+- parameters
+- model
+- context
 links: []
-modified: '2026-04-09T18:07:12+00:00'
+modified: '2026-04-10T01:10:44+00:00'
 summary: Optimal running parameters for NVIDIA's Nemotron 3 Super (120B-A12B MoE),
   covering inference settings for general chat vs tool calling, hardware requirements,
   NoPE context extension, reasoning token handling, GGUF/NVFP4/FP8 formats, and llama.cpp
@@ -87,12 +82,12 @@ NVIDIA recommends two parameter profiles:
 
 **General chat/instruction:**
 ```
-temperature=1.0, top_p=1.0
+temperature=1.0, top_p=1.0, min_p=0.01
 ```
 
 **Tool calling:**
 ```
-temperature=0.6, top_p=0.95
+temperature=0.6, top_p=0.95, min_p=0.01
 ```
 
 ## Hardware Requirements (Total Memory: RAM + VRAM, or Unified Memory)
@@ -133,9 +128,13 @@ Because Nemotron 3 was trained with **NoPE**:
 Nemotron 3 uses ChatML-style format:
 
 ```
-<|im_start|>system\n<|im_end|>\n
+system\n</think>
-<|im_start|>user\n{message}<|im_end|>\n
+
-<|im_start|>assistant\n<think)>{reasoning}</think)>{answer}<|im_end|>\n
+user\n{message}
+</think>
+
+
+<think)>{reasoning}</think){answer}
 ```
 
 ### Reasoning Tokens
@@ -144,6 +143,38 @@ Nemotron 3 uses ChatML-style format:
 - `</think)>` = token ID **13**
 - Use `--special` to see tokens, `--verbose-prompt` to see the prepended `<think)>`.
 
+## Deployment Issues: EOS Token Leak
+
+**Problem:** NVIDIA Nemotron 3 models (Super 120B, Nano 30B) use ChatML-style chat templates. The EOS token `</think>` can leak into generated content when running with the `--special` flag in llama.cpp. This causes parse failures on subsequent turns, e.g.:
+
+```
+Error: Failed to parse input at pos 285: 
+</think>
+```
+
+The `--special` flag is required for Nemotron reasoning tokens, so it cannot be removed.
+
+**Fix:** Add `--stop "</think>"` to the `llama-server` command in the llama-swap config. This tells the server to treat `</think>` as a stop sequence, preventing it from appearing in generated output.
+
+**Applies to:**
+- `nemotron-3-super-120b` — the 120B-A12B flagship model
+- `nemotron-3-nano-30b` — the 30B-A3B compact model
+
+**Config file:** `configs/pfi-ana/docker/conf/llama-swap/config.yaml`
+
+**Example fix in llama-server command:**
+```bash
+./llama.cpp/llama-server \
+    --model unsloth/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF/UD-Q4_K_XL/NVIDIA-Nemotron-3-Super-120B-A12B-UD-Q4_K_XL-00001-of-00003.gguf \
+    --alias "unsloth/NVIDIA-Nemotron-3-Super-120B-A12B" \
+    --prio 3 \
+    --min_p 0.01 \
+    --temp 0.6 --top-p 0.95 \
+    --ctx-size 16384 \
+    --stop "</think>" \
+    --port 8001
+```
+
 ## Example llama.cpp Commands
 
 ### General Instruction (4-bit)