From 1b5d6ba23a3cd64908091dd2e7634be86e63c6e6 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Mon, 14 Sep 2026 02:51:54 -0700 Subject: [PATCH] =?UTF-8?q?docs(flash-next-seat):=20dealignai=20weights=20?= =?UTF-8?q?deleted=20=E2=80=94=20record=20that=20no=20local=20rollback=20e?= =?UTF-8?q?xists?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Operator instruction: delete the displaced dealignai checkpoint. 125 GiB reclaimed from /tank (59% -> 57% used). Verified before removing: not mounted by any running or exited container, no symlinks, no inodes shared with the converted orcarouter directory. Every "rollback is two .env keys" statement across the stack README, the .env.example, persistent-memory and its detail file was true when written and is false now -- the .env backup still names paths that no longer exist. Corrected in place rather than left as false reassurance, since a stale rollback instruction is discovered precisely when it is needed. Reverting this seat now costs a 126 GiB re-download. The quality A/B against dealignai is likewise no longer runnable locally: its reference arm is gone. The pristine 170 GiB orcarouter download is retained deliberately -- it is what makes the PLE bf16->FP8 conversion reproducible without re-fetching -- and that is now recorded so a future session does not reclaim it as an obvious duplicate. Also notes that ~75 GiB of non-PLE shards are duplicated between the pristine and converted orca directories (the convert's hardlinks hit EXDEV across two container bind mounts); both now sit directly on /tank, so relinking would reclaim it if /tank ever tightens. --- .../2026-09-14-fv-seat-reorg-and-orca-blocker.md | 9 ++++++++- persistent-memory.md | 7 +++++-- stacks/flash-next-seat/.env.example | 7 +++++-- stacks/flash-next-seat/README.md | 13 ++++++++++--- 4 files changed, 28 insertions(+), 8 deletions(-) diff --git a/persistent-memory.d/2026-09-14-fv-seat-reorg-and-orca-blocker.md b/persistent-memory.d/2026-09-14-fv-seat-reorg-and-orca-blocker.md index 7a82b59..62f4555 100644 --- a/persistent-memory.d/2026-09-14-fv-seat-reorg-and-orca-blocker.md +++ b/persistent-memory.d/2026-09-14-fv-seat-reorg-and-orca-blocker.md @@ -101,7 +101,14 @@ cost decode speed, which was the standing risk of giving up FP4 tensor-core comp - Quality A/B orca vs dealignai (the actual reason for the swap — the W4A4 long-context degradation axis). Needs a controlled harness + noise floor. - Deep-prefill probe at 262K against THIS checkpoint. Startup is not a depth test. -- Rollback is two `.env` keys; `.env.bak-preorca-20260914-023408` on the host. +- ⚠⚠ **NO LOCAL ROLLBACK.** dealignai weights DELETED 2026-09-14 on operator instruction + (125 GiB reclaimed). `.env.bak-preorca-20260914-023408` still names the old paths but they + no longer exist — it is a record, not a revert. Reverting = 126 GiB re-download. + The pristine 170 GiB `qwen38-flash-next-orcarouter-nvfp4` IS retained (redo the conversion + from it; do not delete it without a reason). +- Disk: ~75 GiB of non-PLE shards are duplicated between the pristine and converted orca dirs + (the convert's hardlinks hit EXDEV inside the container). Now that both sit directly on + /tank, relinking them would reclaim it — not urgent at 3.8T free. - Disk: the convert **copied** ~75 GiB of unchanged shards because hardlinks hit `EXDEV` (separate bind mounts of the same fs). Harmless; reclaimable by relinking if /tank tightens. diff --git a/persistent-memory.md b/persistent-memory.md index b56d8ca..e123746 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -162,8 +162,11 @@ KV 344,155 tokens @ 262,144 ctx, 1.31x concurrency. ⚠ The reorg note's dealign (~121 tok/s) came from a DIFFERENT harness — do not treat this as a measured win over dealignai; a controlled A/B needs dealignai back on this same harness plus a noise floor. -⚠ Not yet done: quality A/B vs dealignai, and a deep-prefill probe at 262K on this -checkpoint. `.env` rollback is two keys (`.env.bak-preorca-20260914-023408`). +⚠⚠ **NO LOCAL ROLLBACK — dealignai weights DELETED 2026-09-14** on operator instruction +(125 GiB reclaimed, /tank 57% used). Reverting = re-downloading 126 GiB. The `.env` backup +names paths that no longer exist. ⚠ Not yet done, and now unbacked: quality A/B vs dealignai +(its reference arm is gone) and a 262K deep-prefill probe. The pristine 170 GiB orcarouter +download is RETAINED — it is what makes the PLE conversion redoable without re-fetching. → `persistent-memory.d/2026-09-14-fv-seat-reorg-and-orca-blocker.md` ### fv-ml1 seat topology now diff --git a/stacks/flash-next-seat/.env.example b/stacks/flash-next-seat/.env.example index 2196f84..38e9b0c 100644 --- a/stacks/flash-next-seat/.env.example +++ b/stacks/flash-next-seat/.env.example @@ -27,8 +27,11 @@ FN_CONTAINER_NAME=vllm-flash-next # in-house (2026-09-14) so vLLM's from_quant_config branch 1 selects the FP8 PLE method. # ⚠ compressed-tensors, NOT modelopt_fp4 — this checkpoint is weight-only on BOTH axes # (W8 float attn / W4 float experts, input_activations: null), which is why it was chosen -# over the dealignai W4A4 build. Rollback: the dealignai pair below, both still on disk. -# FN_MODEL=/tank/aimodels/qwen38-flash-next-abliterated-nvfp4 FN_QUANT=modelopt_fp4 +# over the dealignai W4A4 build. +# ⚠⚠ NO LOCAL ROLLBACK: the dealignai weights were DELETED 2026-09-14 (operator instruction). +# Reverting means re-downloading 126 GiB of dealignai/Qwen3.8-Flash-Next-ABLITERATED-NVFP4. +# The pristine 170 GiB orcarouter source is kept at qwen38-flash-next-orcarouter-nvfp4 so the +# PLE bf16→FP8 conversion can be redone without re-fetching. FN_MODEL=/tank/aimodels/qwen38-flash-next-orcarouter-nvfp4-plefp8 FN_QUANT=compressed-tensors FN_SERVED_NAME=qwen3.8-flash-next-uncensored diff --git a/stacks/flash-next-seat/README.md b/stacks/flash-next-seat/README.md index 8d6d0ff..3e35127 100644 --- a/stacks/flash-next-seat/README.md +++ b/stacks/flash-next-seat/README.md @@ -99,9 +99,16 @@ zero saturation — and the same FP8-PLE treatment dealignai already shipped, so regression against the seat it replaced. `weight_scale` is written BF16 [1] to match the published format. MTP head (31 tensors) and the vision tower carry through untouched. -⚠ Still unmeasured: a controlled quality A/B against dealignai — which is the entire reason -for the swap — and a deep-prefill probe at 262K on this checkpoint. Rollback is two `.env` -keys; the dealignai checkpoint is still on disk. +⚠⚠ **THERE IS NO LOCAL ROLLBACK.** The dealignai checkpoint was deleted on operator +instruction 2026-09-14 (125 GiB reclaimed). Reverting this seat now means **re-downloading +126 GiB** from `dealignai/Qwen3.8-Flash-Next-ABLITERATED-NVFP4`, not flipping two `.env` +keys. The `.env` backup (`.env.bak-preorca-20260914-023408`) still names the old paths, but +those paths no longer exist — treat it as a record of the old settings, not a working revert. + +⚠ Still unmeasured, and now unbacked: a controlled quality A/B against dealignai — which was +the entire reason for the swap, and whose reference arm is gone — and a deep-prefill probe at +262K on this checkpoint. The 170 GiB pristine `qwen38-flash-next-orcarouter-nvfp4` download is +retained; it is what lets the PLE conversion be redone without re-fetching. Rejected alternatives, for the record: `nvidia/…-NVFP4` is the cleanest ModelOpt build but is not abliterated; `lovedheart/…-Pruned-RTXPRO-6000` prunes to 448 of 512 experts;