Update: reference/nemotron-3-super-running-parameters.md — Corrected EOS token leak fix: clarified that --stop flag does not exist in llama-server; stop sequences must be provided in the API payload's 'stop' array instead of as a command-line argument.

reference/nemotron-3-super-running-parameters.md

@@ -5,20 +5,20 @@ keywords:
 - nemotron
 - unsloth
 - llama.cpp
-- nvidia-nemotron-3-super-120b-a12b-gguf
 - nvidia
 - super
-- ud-q4_k_xl
+- nvidia-nemotron-3-super-120b-a12b-gguf
 - reasoning
-- min_p
+- ud-q4_k_xl
 - nvidia-nemotron-3-super-120b-a12b
+- model
 - huggingface.co
 - running
 - parameters
-- model
 - context
+- top_p
 links: []
-modified: '2026-04-10T01:10:44+00:00'
+modified: '2026-04-10T01:48:14+00:00'
 summary: Optimal running parameters for NVIDIA's Nemotron 3 Super (120B-A12B MoE),
   covering inference settings for general chat vs tool calling, hardware requirements,
   NoPE context extension, reasoning token handling, GGUF/NVFP4/FP8 formats, and llama.cpp
@@ -131,10 +131,7 @@ Nemotron 3 uses ChatML-style format:
 system\n</think>
 
 user\n{message}
-</think>
+</think>{reasoning}</think){answer}
-
-
-<think)>{reasoning}</think){answer}
 ```
 
 ### Reasoning Tokens
@@ -154,7 +151,7 @@ Error: Failed to parse input at pos 285:
 
 The `--special` flag is required for Nemotron reasoning tokens, so it cannot be removed.
 
-**Fix:** Add `--stop "</think>"` to the `llama-server` command in the llama-swap config. This tells the server to treat `</think>` as a stop sequence, preventing it from appearing in generated output.
+**Fix:** Stop the generation when the `</think>` token appears by including it in the **payload's `stop` array** when calling the API. The `--stop` command-line flag does not exist in llama-server; stop sequences must be handled at the request level in the API payload.
 
 **Applies to:**
 - `nemotron-3-super-120b` — the 120B-A12B flagship model
@@ -162,17 +159,24 @@ The `--special` flag is required for Nemotron reasoning tokens, so it cannot be
 
 **Config file:** `configs/pfi-ana/docker/conf/llama-swap/config.yaml`
 
-**Example fix in llama-server command:**
+**Example fix in API payload (Python client):**
+```python
+completion = openai_client.chat.completions.create(
+    model = "unsloth/NVIDIA-Nemotron-3-Super-120B-A12B",
+    messages = [{"role": "user", "content": "What is 2+2?"}],
+    stop=["</think>"]  # Stop sequences must be in the payload, not command-line
+)
+```
+
+**Example fix with cURL:**
 ```bash
-./llama.cpp/llama-server \
+curl -X POST http://localhost:8001/v1/chat/completions \
-    --model unsloth/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF/UD-Q4_K_XL/NVIDIA-Nemotron-3-Super-120B-A12B-UD-Q4_K_XL-00001-of-00003.gguf \
+    -H "Content-Type: application/json" \
-    --alias "unsloth/NVIDIA-Nemotron-3-Super-120B-A12B" \
+    -d '{
-    --prio 3 \
+        "model": "unsloth/NVIDIA-Nemotron-3-Super-120B-A12B",
-    --min_p 0.01 \
+        "messages": [{"role": "user", "content": "What is 2+2?"}],
-    --temp 0.6 --top-p 0.95 \
+        "stop": ["</think>"]
-    --ctx-size 16384 \
+    }'
-    --stop "</think>" \
-    --port 8001
 ```
 
 ## Example llama.cpp Commands
@@ -232,7 +236,7 @@ openai_client = OpenAI(
 
 completion = openai_client.chat.completions.create(
     model = "unsloth/NVIDIA-Nemotron-3-Super-120B-A12B",
-    messages = [{"role": "user", "content": "What is 2+2?"}],
+    messages = [{"role": "user", "content": "What is 2+2?"}]
 )
 
 # Access both reasoning and final answer