vh
1a5bc2ddf1
The v2 dense quant had failed four times. Attempt 5 lands it at 19 G. The blocker was not what it looked like. `AmbiguousGlobalPerLayerAttributeError` on `head_dim` read as a malformed upload -- DogOnKeyboard's config carries a `per_layer_config` key zerofata's canonical one lacks -- and the standing fix was to force `allow_global_per_layer_attribute_access=True`. Both halves were wrong. `pip install llmcompressor==0.13.0` downgrades transformers 5.16.1 -> 5.14.1. The config was serialized by 5.16.1, which materializes `per_layer_config` from `global_head_dim` + `layer_types`; 5.14.1 carries the heterogeneity guard but not the gemma4 resolver. Under the image's own transformers the same config loads fine. `:latest` was also re-pulled during attempt 4 and no earlier run, so the toolchain moved mid-diagnosis. Two things separated "malformed upload" from "moved toolchain": reproducing the real failing call (a bare AutoConfig load does not reproduce it; the trigger is reached through AutoTokenizer) and keeping zerofata's canonical tree, quantized cleanly on 2026-08-21, as a positive control. The fix drops `per_layer_config` rather than forcing global access. It is exactly redundant -- keys are precisely the ten full_attention layer indices, sole value (512, 4), verbatim the global fields -- and forcing instead would make `config.head_dim` answer 256 to the callers building the 512-wide layers. patch_perlayer.py re-proves that redundancy at apply time and refuses if it ever stops holding. Verified on the tensor table rather than the exit code: the output is identical family-for-family and count-for-count to the August canonical quant, with 356 BF16 vision-tower tensors preserved and input_activations=None. A GPU-free load leaves 0 tensors on meta and generates coherent prose. The section 4.4 serve test has NOT run -- GPU1 has 19.9 GB free against 19.5 GB of weights, so it needs a live seat displaced. Also fixes the A4B output, which had a truncation cap baked into its tokenizer (max_length 8192) from being quantized with the calibration corpus. Playbook gains section 3.17 for the pinned-transformers class and sharpens 3.16 to say drop the dataset outright for any A16 scheme.
docs/
Navigation map for the documentation tree. New session? Read
orientation.md first — it's the narrative overview
of the fleet, backup architecture, governing principles, and gotchas,
and it points at everything else.
Tree
docs/
├── orientation.md # start here — fleet overview + where-to-look guide
├── runbooks/ # ops runbooks (recovery, deployment phases)
│ ├── disaster-recovery.md
│ ├── nh3-prune-ritual.md
│ └── pbs-deployment.md
└── pfi/ # PFI-specific reference (services, models, VMs)
├── docker-stack.md
├── model-list.md
├── proxmox-vms.md
├── recommended-model-settings.md
├── vm-102-matrix-appservice.md
└── vm-102-matrix-synapse.md
What goes where
runbooks/— step-by-step ops procedures. Anything you'd reach for during an incident or while standing up new infrastructure. Examples: disaster recovery (blast-radius tiers + restoration steps), PBS deployment (9-phase rollout). New runbook → new file here.pfi/— PFI-specific reference material that's too narrow for the top-level CLAUDE.md but doesn't change incident response. AI model inventory, recommended inference settings, Matrix bridge config, Proxmox VM map. New stable reference → new file here.- Top-level (
docs/orientation.md,docs/README.md) — narrative guides about the workspace itself, not about specific infra.
Cross-references
- Fleet topology + servers table: top-level
CLAUDE.md. - Open work + recent milestones: top-level
STATUS.md. - Durable cross-session facts:
~/.claude/projects/-home-lkraven-development-eshpfi-management/memory/.
Conventions
- Markdown, GitHub-flavored. CommonMark renders fine in most viewers.
- File names are lowercase-kebab-case, descriptive. No dates in filenames — git history covers that.
- One topic per file. If a file grows past ~500 lines, look for a natural split before adding more.
- No checked-in binaries or checksums. Build/release artifacts belong
in a build pipeline or
tools/, notdocs/.