Commit Graph
3 Commits
Author SHA1 Message Date
vh 196416f3da docs(cyberprev-seat): correct model lineage — it is a cyber SFT finetune, not an abliteration
Earlier commits and the compose header called cyberprev "an abliteration of
Qwen3.8-27B (refusal-direction weight edit, no finetune)". That was wrong. Per
the hotdogs model card (base_model_relation: finetune), cyberprev is an
offensive-security agentic tool-calling LoRA, merged at scale 1.0, trained on
hotdogs/cyber-sft-agent-qwen38 (8,400 rows across 22 pentest tools: nmap,
sqlmap, metasploit, hydra, crackmapexec, ...). The abliteration is INHERITED
from its base (hotdogs/Qwen3.8-27B-abliterated); the new capability in this
repo is the cyber SFT itself.

So the sec-seat lineage is: mog-sec = persona on stock weights; sentinel-r3 =
pentest-trajectory SFT on stock weights; cyberprev = cyber tool-calling SFT on
an abliterated base. All three are SFT-or-prompt variants; the earlier framing
of cyberprev as "not a finetune" was backwards.

Corrects the compose header lineage block, the homepage dashboard description,
and the gateway alias comment. No functional change; the running seat and its
weights are unchanged (label refreshes on next container recreate).
2026-09-14 07:28:26 -07:00
vh b94d173887 fix(cyberprev-seat): serve coherent v2 quant from name-repaired source; wire gateway
The first cyberprev quant served pure gibberish despite passing every structural
check. Root cause: the hotdogs/Qwen3.8-27B-abliterated-cyber-preview checkpoint
ships broken -- 850 of 1199 tensors (the whole transformer body) carry a TRIPLED
`language_model.` prefix, confirmed present in the upstream HF index (an unsloth
export bug, not our download). transformers silently loaded the unmapped tensors
as random-init, the pipeline quantized noise, and structural verification passed
because it inspects the output config, never that input names mapped. A plain
greedy coherence smoke caught it; nothing structural could.

Fix: rewrite tensor names against a known-good reference of the same architecture
(qwen38-27b-uncensored-bf16), verified the repaired name-set == reference name-set
exactly (1199 tensors, 0 collisions), and re-quantized from the corrected source
(cyberprev-bf16-fixed). v2 is coherence-verified (spec off AND live with dflash):
Paris/Berlin/Rome, correct arithmetic precedence, primes, clean cyber-domain chat
with a proper thinking split. dflash k=7 measured at 2.77 mean acceptance length.

Seat serves cyberprev-27b / cyberprev-27b-thinking on :8025/GPU0 with dflash k=7,
displacing the retired sentinel-r3. Gateway: removed the retired sentinel-r3 /
sentinel-r3-reasoning aliases and added cyberprev-27b / cyberprev-reasoning -- NOT
repointed, since serving cyberprev weights under sentinel's name is silent
substitution. Verified end-to-end through the gateway; sentinel-r3 now 400s.

Playbook gains a superseded-claims row: structural verification does not imply a
good quant; a coherence smoke is the load-bearing gate, and third-party sources
must have their tensor names diffed against a reference before quantizing.

Broken v1 quant dir removed.
2026-09-14 04:28:59 -07:00
vh 1418edb03e feat(cyberprev-seat): mixed-NVFP4 quant of the abliterated cyber-preview, displacing sentinel-r3
Third sec-seat candidate: hotdogs/Qwen3.8-27B-abliterated-cyber-preview, an
abliteration (refusal-direction weight edit) of Qwen3.8-27B aimed at the
cyber-offense refusal surface -- distinct from mog-sec (persona on stock
weights) and sentinel-r3 (SFT finetune). Operator instruction: quant it, take
sentinel down, serve it with mtp or dflash.

Quantized to the house mixed recipe via services/gen-seat-mixed-quant/ (NVFP4
W4A4 on MLP layers 0-55 + FP8 W8A8 on attn/linear_attn/lm_head/MLP 56-63, FP8
KV). The prior attempt (2026-09-11/14) died with "Cannot determine
num_attention_heads" because it ran from a bare .venv whose newer
compressed-tensors reads that field at top level; quant_mixed_nvfp4.py already
promotes text_config attention fields for exactly this reason, and the run
through the canonical vLLM-image + llmcompressor 0.13.0 / compressed-tensors
0.18.0 path (versions recorded from the container) completed clean.

post_quant.py did its job: grafted the 15 MTP tensors verbatim (BF16), and
re-injected re:^mtp.* into the ignore list after llm-compressor pruned it for
matching no module at quant time -- without which vLLM loads the grafted head
uninitialised and speculative decoding runs at 0% acceptance. It also caught a
missing preprocessor_config.json (absent from the abliterated source AND its
hotdogs upstream); restored from Qwen/Qwen3.8-27B, verified byte-identical to
the working sentinel-r3 build, so the vision tower preprocesses.

Verified from the tensors, not the config: NVFP4 covers MLP 0-55, FP8 covers
56-63, no overlap, 168 weight_packed tensors (56x3), 15 BF16 MTP, 333 BF16
vision. 51.0 GiB bf16 -> 21.0 GiB.

Served under its own name (cyberprev-27b / -thinking), NOT sentinel-r3 --
serving different weights under a retired name is silent substitution. Takes
over :8025 and GPU 0 (co-resident with mog-sec). dflash k=7 is the default,
carried from the sentinel measurement, but is re-measured against MTP on this
ABLITERATED body before cutover, since abliteration is exactly what can desync
an MTP head. Drafter (qwen38-27b-dflash2-drafter) shared with mog-sec.

Context capped at 163840 not native 262K, inherited from mog-sec's hard-won
depth lesson; raise only after a non-repeating deep-prefill probe on this
checkpoint.
2026-09-14 03:32:09 -07:00