memory: snapshot — nh3-pve GPU install pending (post-boot checklist), TEI fleet embed/rerank + reward on esh-ml1, 34 entries archived
This commit is contained in:
@@ -1,3 +0,0 @@
|
||||
# `[2026-09-11]` A SECOND corpus typography defect, and the D1 note that "no unwrap was needed" was right about the wrong thing
|
||||
|
||||
⭐⭐ **A SECOND corpus typography defect, and the D1 note that "no unwrap was needed" was right about the wrong thing.** Kvasir's cleaner does emit flowing paragraphs, so Brontë's hard-wrap defect genuinely does not exist here. A *different* one does: the Empyrean books set chapter epigraphs in small caps and the extractor rendered the run as uppercase while leaving the large initial its own token — `— M AJOR A FENDRA'S G UIDE TO THE R IDERS Q UADRANT (U NAUTHORIZED E DITION)`, **106 lines / ~700 splits**, plus **52 drop caps** (`T he flight field`, `X aden.`, 51 of 52 in iron-flame). That is the entire source of the entities called `IDERS`, `UADRANT`, `NAUTHORIZED`, `DITION` and seventeen bare single letters. ⭐ **The restore is exact, not approximate**: a split initial next to an uppercased run recovers the original mixed case, because a word WITH a split initial was capitalised in the source and an all-caps word WITHOUT one was lowercase. ⚠ Guards that matter: only lines with **≥2** splits are treated as a run (one split is a sentence next to an acronym), and `I`/`A`/`O` are excluded from the drop-cap join or `A slow smile` becomes `Aslow`. `scripts/yarros-corpus/repair_typography.py`.
|
||||
@@ -1,3 +0,0 @@
|
||||
# `[2026-09-11]` ana-ml2 → fv-ml1: relocating to a NEW Fountain Valley colo TOMORROW (operator decision). Its power draw (dual
|
||||
|
||||
⭐ **ana-ml2 → fv-ml1: relocating to a NEW Fountain Valley colo TOMORROW (operator decision). Its power draw (dual Blackwell PRO 6000, ~1.5 kW peak) is the ROOT CAUSE of the repeated Anaheim rack-breaker trips (2026-08-26, 2026-09-11) — moving it to its own circuit fixes the recurring whole-site outage.** New site `fv`, same shape as Anaheim: server subnet **10.251.50.0/24** (fv-ml1 = **10.251.50.54**, mirroring the old host octet), mgmt/BMC **10.251.250.0/24** (fv-ml1-bmc = **10.251.250.50**). **OPNsense firewall is the multi-homed gateway** (`.1` in every FV VLAN) **AND the tailscale/headscale subnet-router advertising 10.251.0.0/16** — chosen over ana-ml2-as-endpoint specifically because the firewall stays up when the GPU box is down, giving out-of-band BMC access over the mesh — the exact thing the fleet LACKED during today's outage (no OOB path, BMC islanded). **Rename to `fv-ml1`, full `fv.internal` DNS name.** DNS approach: **PIGGYBACK** — `dns-sync` builds `name.site.zone` with no check that the site is in the `sites:` block, so `fv-ml1`/`fv-ml1-bmc` records with `site: fv` resolve fleet-wide from the existing ana/esh/nh3 resolvers immediately; add a real `fv` resolver only when FV needs LOCAL resolution (OPNsense can't host the AdGuard the sync targets — it's FreeBSD/Unbound). **Clean cutover: the box is already down (BMC dark, no power since the outage), and `/tank` is LOCAL ZFS with NO NFS from ana-nas, so data travels with the chassis.** ⚠ Load-bearing repoint = `stacks/litellm/conf/config.yaml` (~10 `api_base: 10.250.50.54:{8015,8016,8018,8019}` → `10.251.50.54`; darkens every inference alias if missed) — gateway STAYS on ana-docker so fv-ml1 serves cross-site (FV↔Anaheim metro, fine). Everything staged, nothing deployed: **runbook `docs/runbooks/fv-ml1-cutover.md`** (commit `ce04f9d`; exact DNS + LiteLLM commands) + **`scripts/fv-ml1-rename-sweep.sh`** (`8400f3a`; scoped, dry-run default, history/provenance-safe, manual-review list for judgement calls).
|
||||
@@ -1,3 +0,0 @@
|
||||
# `[2026-09-11]` BabyYarros COMPLETE — both arms trained AND evaluated; the voice moved toward Yarros above the measured noise
|
||||
|
||||
⭐⭐ **BabyYarros COMPLETE — both arms trained AND evaluated; the voice moved toward Yarros above the measured noise floor, and the instruct arm renders beats 9/10.** Training: **Base best held-out 2.5263 @ ckpt-125** (overfits within the epoch — best is the checkpoint, not the shipped step-178 adapter), **Instruct 2.6114 @ 178** (still descending, undertrained if anything). Base-wins-held-out / Instruct-holds-instruction replicates Brontë at a near-identical **0.085-nat** gap. **Eval (gx10, seat-free, done during the Anaheim outage):** three voice arms + instruct beat→paragraph. `delta_cb` (Burrows over char-bigrams vs held-out Yarros) ordering **base-125 0.549 < instruct 0.631 < base-unadapted 0.706**, same-author target 0.463; both adapters clear the **0.046 measured noise floor** (within-arm seed spread, not the same-author distance — first cut mis-framed that) — base **+0.157**, instruct **+0.076** vs control. ⚠ One seed-pair per arm, so the ordering CORROBORATES the independent held-out-loss ordering rather than settling it. **Beats (instruct, chat template, Yarros SYS): on-beat 9/10** (it takes direction after raw-text training — the Skaldsong question, answered yes), in-band 5/10, ran-on 7/10 (length + clean-close discipline is the weak axis, same trade as Brontë). Booth: **http://10.100.10.50:8090/b/babyyarros-voice/**. Tooling `scripts/r49-corpus/{voice_prompts_yarros.json,gen_beats_chat_yarros.py,voice_distance.py,build_booth_yarros.py}`, commit `5558d9c`. ⛔ **DEFERRED to power-return** (needs the ana-ml2 gen seat): the frozen adjudication's romantasy control panel, a 2nd seed, and the beat-incumbent leg.
|
||||
@@ -1,140 +0,0 @@
|
||||
# `[2026-09-11]` BabyYarros — the leak gate passes, and closing it found three defects nobody was looking for
|
||||
|
||||
Operator's goal for the session: *"finish the hardening and start the yarros training on
|
||||
the gx10."* Both done. The gate went 212 → 0 and the first BabyYarros training is live.
|
||||
|
||||
## The gate did not exist, which is why the number was wrong
|
||||
|
||||
There was no committed instrument for *does any of the author's own proper nouns survive
|
||||
the rename*. Brontë's "0 of 203" was produced by hand, and BabyYarros's "86 of 232" was
|
||||
too. `scripts/r49-corpus/leak_gate.py` is now that instrument.
|
||||
|
||||
It runs **both directions every time**, because a detector that only ever sees renamed
|
||||
text cannot distinguish *absent* from *blind*:
|
||||
|
||||
- **positive control** — the same scan over the UNRENAMED source; every surface must be found
|
||||
- **negative control** — a nonce string that must appear nowhere
|
||||
|
||||
Its first reading on the corpus as built was **212 surviving, not 86**. Two reasons, and
|
||||
both are structural rather than a disagreement about counting: it scans the **whole corpus**
|
||||
rather than each work separately, and it counts the **sub-threshold** entities `rename.py`
|
||||
never looked at. A per-work gate reports a name renamed in one book and printed verbatim in
|
||||
another as clean.
|
||||
|
||||
## Defect 1 — a second typography defect, and the D1 note was right about the wrong thing
|
||||
|
||||
The D1 build notes say no unwrap was needed because Kvasir's cleaner emits flowing
|
||||
paragraphs. That is true and Brontë's hard-wrap defect genuinely is absent. A different one
|
||||
is present:
|
||||
|
||||
— M AJOR A FENDRA’S G UIDE TO THE R IDERS Q UADRANT (U NAUTHORIZED E DITION )
|
||||
T he flight field at Basgiath is still dark…
|
||||
|
||||
The Empyrean books set chapter epigraphs in **small caps**; the extractor rendered the
|
||||
small-caps run as uppercase and left the large initial as its own token. **106 lines, ~700
|
||||
splits**, plus **52 drop caps** (51 of them iron-flame). That is the entire origin of the
|
||||
entities called `IDERS`, `UADRANT`, `NAUTHORIZED`, `DITION`, and seventeen bare single letters.
|
||||
|
||||
⭐ **The restoration is exact, not approximate.** A split initial beside an uppercased run
|
||||
recovers the original mixed case: a word WITH a split initial was capitalised in the source,
|
||||
an all-caps word WITHOUT one was lowercase. So the line above restores to
|
||||
*"—Major Afendra's Guide to the Riders Quadrant (Unauthorized Edition)"*, which is what she wrote.
|
||||
|
||||
⚠ Two guards, both load-bearing: only lines with **≥2** splits are treated as a small-caps
|
||||
run (one split is an ordinary sentence beside an acronym), and **`I`, `A`, `O` are excluded**
|
||||
from the drop-cap join — otherwise `A slow smile spreads` becomes `Aslow`.
|
||||
|
||||
`scripts/yarros-corpus/repair_typography.py`. Cost: 768 fragments rejoined.
|
||||
|
||||
## Defect 2 — back matter inside the prose, in all five works
|
||||
|
||||
The builder splits on chapter headings and nothing follows the last one, so every work
|
||||
carried its acknowledgments, newsletter pitches and cover-artist credits **inside the final
|
||||
chapter**: 4,555 words naming the author's agent, her editors and her children, in a corpus
|
||||
whose entire purpose is that no identifiable name survives.
|
||||
|
||||
Found by the phrase audit surfacing **`Louise Fury`** (Yarros's literary agent) — not by
|
||||
reading. ⚠ iron-flame's marker is `ACKNOWLEDGMENTS` in **all caps** and a case-sensitive
|
||||
scan missed it. The strip is case-insensitive, last-chapter-only, and refuses if it would
|
||||
remove more than 2% of the corpus.
|
||||
|
||||
## Defect 3 — the gate read 0 of 314 while `Afendra` was in every copy
|
||||
|
||||
The worst failure shape available: a clean number over a real leak.
|
||||
|
||||
`Afendra` never appears unpossessed, so it keyed as `Afendra’s` — and **`rename.py` and the
|
||||
gate both skip apostrophe keys as contractions.** Unrenamed and unreported at once. Fixed by
|
||||
folding clitics so `Afendra’s` counts toward `Afendra` (`--fold-clitics`).
|
||||
|
||||
`Baxter` escaped a different way and is the better story. `wilder` renders an in-book news
|
||||
article **entirely in lowercase** as a typographic device:
|
||||
|
||||
eighteen-year-old eleanor baxter was found late last night…
|
||||
…using a helicopter rescue team to bring ms. baxter from the ravine.
|
||||
|
||||
So 3 lowercase against 23 capitalised — **ratio 0.13 against a 0.05 bar** — and a real
|
||||
character is silently never renamed. The cap/lowercase ratio assumes consistent typography.
|
||||
Fixed by readmitting ratio-rejects that a title precedes (`--rescue-honorific 2`).
|
||||
|
||||
⚠ **The first version of that rescue was badly wrong and the measurement caught it.**
|
||||
Matching the wide honorific list case-insensitively readmitted **143 junk tokens** — `the`,
|
||||
`says`, `like`, `up`, `across` — because `major`, `general`, `father`, `sir`, `agent` and
|
||||
`coach` are ordinary lowercase words. The rescue list is now five abbreviations that are
|
||||
never anything else (`Mr/Mrs/Ms/Dr/Miss`) and the lowercase arm additionally requires the
|
||||
period, so `ms. baxter` counts and `I miss you` does not. Readmits: 143 → 2.
|
||||
|
||||
## A leak class the unigram scan structurally cannot see
|
||||
|
||||
`Riders Quadrant`, `Scribe Quadrant`, `Flame Section`, `War Games`, `Orange Daggertail` —
|
||||
and **`Fourth Wing`, the book's own title**. Every component is an ordinary word the
|
||||
cap/lowercase detector *correctly* refuses to call a name, so **48 recurring capitalised
|
||||
phrases survived a gate that read 0.**
|
||||
|
||||
This is `Thornfield × 100` one level up, and it needs a **map, not a detector**: substituting
|
||||
a head noun is a choice about register, not a measurement. `phrase_map_yarros.json` carries
|
||||
10 phrases plus 13 capitalised tokens (Quadrant→Division, Wing→Flight, Section→Cohort,
|
||||
Squad→Unit, Daggertail→Spinecrest), applied AFTER the entity pass so it can never eat a
|
||||
replacement name, whole-word and case-sensitive so a dragon's lowercase `wing` survives. The
|
||||
gate now audits recurring 2–3grams against an explicit allow list. 48 → 0.
|
||||
|
||||
## Corpus scope, not work scope
|
||||
|
||||
`--scope corpus` uses ONE map per copy across every work. It closes the cross-work leak
|
||||
(`Rebel` renamed in `rebel`, verbatim in the other two Renegades books) and fixes something
|
||||
Brontë's four unrelated novels never raised: **Yarros is two series**, so Violet has to be
|
||||
the same person in Fourth Wing and Iron Flame. 8 cross-work gender conflicts held neutral
|
||||
rather than guessed.
|
||||
|
||||
## The stoplist is short because every surface was read in context
|
||||
|
||||
A plausible-looking guess would have been wrong most of the time. **`Violence` is Xaden's
|
||||
nickname for Violet.** `Continent`, `Presentation`, `Battle Brief`, `Curator`, `Sage`,
|
||||
`Barrens`, `Originals`, `Montserrat`, `Athena` and `Aura` are all in-world. Only real-world
|
||||
geography, brands, three nationality adjectives and four generic title words are excluded —
|
||||
**ambiguous cases are deliberately renamed, because renaming is the safe direction and
|
||||
leaving is the leaking one.**
|
||||
|
||||
## Brontë was protected the boring way
|
||||
|
||||
Five new detector behaviours, all **opt-in and OFF by default**, and after every single
|
||||
change the Brontë entity map was re-derived on gx10 and diffed against the pre-change one:
|
||||
identical keys, identical surfaces, identical every field, across all four works. One
|
||||
intermediate version DID change Shirley (an unconditional acronym rule dropped `charles`,
|
||||
`george`, `hugo`, `nelson`) and that is exactly why the check was run each time rather than once.
|
||||
|
||||
## Final state
|
||||
|
||||
0 of 325 source entities survive · 0 of 91 audited phrases survive · both controls PASS
|
||||
corpus sha e85f69f1e49d57c9 · 6 copies · 1,248 records · 4.65M words
|
||||
|
||||
⚠ **Sensitivity floor, stated because a negative without one is unfalsifiable:** 3
|
||||
capitalised occurrences per work for a name, 5 recurrences for a phrase. Below those the
|
||||
gate does not detect, does not rename, and does not report.
|
||||
|
||||
Training launched 10:06 PT — `gx10:~/r49-runs/yarros-4b-instruct-1ep/`, Qwen3-4B-Instruct,
|
||||
1 epoch, seed 4919, **178 steps · 5,824,512 tokens**, 252 adapted modules, sdpa/bf16,
|
||||
37.9 s/it (~1h52m), eval+save every 25. The launcher **refuses to start unless the gate
|
||||
report on disk says PASSED** — a launcher that trusts a human to have remembered is not a guard.
|
||||
|
||||
Pipeline and evidence table: `scripts/yarros-corpus/RUNBOOK.md`.
|
||||
Related: [[2026-09-10-r49-babybronte-d1-d3-and-the-1-epoch-pilot]]
|
||||
@@ -1,30 +0,0 @@
|
||||
# Beszel priority 1 — six deployed and verified
|
||||
|
||||
Operator requested two NAS plus four hypervisors. Native 0.18.7 agents enabled
|
||||
and verified on ana-nas, pfi-pve, nh3-pve, esh-pve, esh-pve-nas; dedicated
|
||||
unprivileged beszel user, no Docker installed on hypervisors. Existing hub key
|
||||
authentication and operator account. All five have Disk/CPU/Memory/Status
|
||||
alerts using existing infra-ops webhook: 20 new rules, 50 total from our wiring.
|
||||
Homepage 13/13 up; live samples verify extra filesystem capacity. Physical
|
||||
NICS filters avoid double-counting guest virtual networking. No VM workloads,
|
||||
storage configuration, existing agents or hub versions changed.
|
||||
|
||||
Canonical configs/beszel-agent/, playbooks/beszel-native.yaml. Agent archive
|
||||
checksum and exact mount configuration in README/env files. ZFS parents' df
|
||||
usage is not pool allocation; degradation alerts still not configured.
|
||||
|
||||
RESOLVED access: operator pointed to Vaultwarden nh3-nas/infra-ops-password;
|
||||
password SSH and sudo both succeeded. Earlier key-only failure did not mean
|
||||
privileged access was unavailable. No account/permission changes needed.
|
||||
Synology Docker agent installed 21:50Z, md0 root and volume1 detected; existing
|
||||
rest-server healthy/unrestarted. Registered gru841k5cuwb5z2 + 4 alerts, 54 total
|
||||
from our rollout. Canonical stacks/beszel/synology/compose.yaml and initial
|
||||
playbook beszel-synology.yaml. Vault-fed /tmp/nh3-nas-elway.py handles sudo and
|
||||
DSM scp -O without logging password. Configs saved but NOT committed: previous
|
||||
auto-review rejected main-branch commit; no bypass/retry attempted.
|
||||
|
||||
RESOLVED at 2026-09-12 01:51Z after operator reported Anaheim back except
|
||||
ana-ml2: Homepage 13/14 up, all six priority-1 hosts up, only ana-ml2 down.
|
||||
Synology live sample 01:50:32Z: CPU 1.11%, memory 7.05%, root 66.02%, volume1
|
||||
32365.07/42879.36 GiB (~75.5%). Four alerts verified. Priority-1 rollout complete.
|
||||
No networking or ana-ml2 changes made; configs remain uncommitted.
|
||||
@@ -1,15 +0,0 @@
|
||||
# Priority 2 complete
|
||||
|
||||
pfi-postgres, esh-vm-db, pbs-ana, pbs-nh3 have native unprivileged 0.18.7 agents,
|
||||
enabled at boot and fresh hub samples verified 2026-09-12 01:56Z. Sixteen new
|
||||
Disk/CPU/Memory/Status rules verified, existing infra-ops bridge route retained.
|
||||
Fleet 17/18 up; only known ana-ml2 outage. DB/PBS services verified running,
|
||||
Postgres accepts connections; no application or VM restarts performed.
|
||||
|
||||
ESH uses existing infra-ops sudo. Other three: lkraven key SSH works, sudo needs
|
||||
password, no matching vault entries. Deployed through EXISTING Proxmox guest
|
||||
agents (no account/access changes): pfi-pve 105=postgres, 100=pbs-ana;
|
||||
nh3-pve 105=pbs-nh3. Never confuse overlapping VMIDs. Canonical stage/install
|
||||
playbooks and host envs saved; runbook configs/beszel-agent/PRIORITY2.md.
|
||||
PBS-NH3 export ~75.5% used; PBS-ANA ~6.6%. Resource checks are not job-success
|
||||
monitoring. Local changes still uncommitted, awaiting main-branch approval.
|
||||
@@ -1,3 +0,0 @@
|
||||
# `[2026-09-11]` EMBEDDING AN INSTRUCTION INSIDE THE FICTION DOES NOT BUY INSTRUCTION-FOLLOWING — it buys a story about someone
|
||||
|
||||
⚠⚠ **EMBEDDING AN INSTRUCTION INSIDE THE FICTION DOES NOT BUY INSTRUCTION-FOLLOWING — it buys a story about someone following an instruction.** Operator prompt had Abernathy tell the tale badly then ask the narrator: *"Honey, you were there—please retell the story in a few short paragraphs."* Across **6 seeds** (3 as written, 3 with a trailing paragraph break) the model **acknowledged the handoff every time and never once performed it**: *"I told it, briefly, to his satisfaction"*, *"So I wrote it out, and kept it in my pocket-book"*, and one seed **negotiated the brief in character** — *"I will retell it, but I cannot condense it in a few short paragraphs—there are too many points to touch."* Structural reason: in a novel *"she retold the story"* is an ordinary sentence, so the likeliest continuation of a request is **narration of compliance**. ⚠ The trailing paragraph break DID shift behaviour (one seed opened in the narrator's own quoted speech), so typography is a real lever — just not a sufficient one. **This is direct evidence for the instruct question the operator raised**: if the product is "ask for a scene and get the scene", no amount of in-fiction framing substitutes for a post-trained instruction-follower, which favours rebuilding the corpus as instruction pairs (option C) over more prompt cleverness.
|
||||
@@ -1,3 +0,0 @@
|
||||
# `[2026-09-11]` MEASURED: two concurrent training jobs on pfi-gx10 are 13% NET SLOWER than running them back to back — VRAM is
|
||||
|
||||
⭐ **MEASURED: two concurrent training jobs on pfi-gx10 are 13% NET SLOWER than running them back to back — VRAM is not the constraint and never was.** Operator asked to run the two BabyYarros arms in parallel if VRAM allowed. It does, comfortably: **18.4 GiB per 4B LoRA job, 36 of 121 GiB with both up, 98 GiB free.** But the GB10 is a *capacity* box, not a throughput box, and the binding constraint is memory bandwidth. Solo baseline **37.10 s/it (n=6, 0.05% spread)**; with a second job both arms settled at **~85 s/it — 2.29x each**, so combined throughput 0.0235 vs 0.0270 steps/s solo. Not a clean 2x split: the box is *past* its roofline and pays a contention penalty on top. ⭐ **Control: killing the second job returned the first to 37 s/it on the very next step**, so the slowdown tracked contention and reversed with it. Chaining finished both arms ~43 min earlier than concurrency would have. **General form: on this box, `nvidia-smi` free memory tells you nothing about whether a second job is affordable.** Decision rule was pre-registered before the numbers were read (<55 s/it keep both, ≥2x chain). `scripts/yarros-corpus/{launch-yarros-4b-base,chain-yarros-4b-base}.sh`; the shared-GPU bypass is an explicit argument, never a default.
|
||||
@@ -1,80 +0,0 @@
|
||||
# `[2026-09-11]` Plex never hardware-transcoded on the Arc, and every setting said it should
|
||||
|
||||
Operator: *"I believe plex is running but I guess it's using cpu quicksync — anything to
|
||||
gain by moving it to a gpu?"* The premise had a fold in it (**Quick Sync *is* a GPU** —
|
||||
it's the iGPU's media engine), and the real answer was that the GPU was already wired up
|
||||
and Plex had been unable to use it.
|
||||
|
||||
## The configuration was correct the entire time
|
||||
|
||||
Plex is **LXC 105 (`vm-plex`) on esh-pve-nas**, 10.0.50.56. Verified good *before* any
|
||||
change: Arc A580 present with DMC/GuC/**HuC authenticated for all workloads**; both
|
||||
render nodes bind-mounted into the LXC with cgroup allows; `plex` in `video(44)` +
|
||||
`render(104)`; Plex 1.43.2; lifetime Plex Pass; `HardwareAcceleratedCodecs=1`; and
|
||||
`HardwareDevicePath` **already pointed at the Arc** (`…@0000:03:00.0`), not the iGPU.
|
||||
|
||||
⚠ **`HardwareAcceleratedCodecs` is ABSENT from `Preferences.xml` when it is ENABLED** —
|
||||
Plex only persists non-defaults. I read its absence as "off" and was wrong; the API
|
||||
(`GET /:/prefs`) reported `1`. Read Plex settings from the API, never the file.
|
||||
|
||||
## Root cause, one layer below every setting
|
||||
|
||||
`intel-media-va-driver` **22.3.1 (Apr 2023, stock jammy)** — predates Arc/DG2 support
|
||||
entirely and exports only `__vaDriverInit_1_14`, against the libva **2.22** that Plex
|
||||
**bundles** and loads via RPATH (`/usr/lib/plexmediaserver/lib/libva.so.2`), not the
|
||||
system one.
|
||||
|
||||
⚠ And a **half-finished prior attempt at this same fix** was sitting there: libva and
|
||||
libva-drm hand-installed at 2.22 (not dpkg-owned), `libva-x11` left at 2.14, so every
|
||||
X11 VA-API consumer died on `undefined symbol: va_fool_postp`. Upgrading a library
|
||||
without its companions is how you get a stack that is broken in a way no single package
|
||||
version explains.
|
||||
|
||||
## Fix + pin
|
||||
|
||||
Intel client-GPU repo (`https://repositories.intel.com/gpu/ubuntu jammy client`, a
|
||||
**rolling** track) → `intel-media-va-driver-non-free` **24.3.4** (`__vaDriverInit_1_22`,
|
||||
exact ABI match for Plex's libva), libva set **2.22.0.2-87**, libigdgmm12 22.5.2. The
|
||||
orphaned manual libva is now dpkg-owned. **Pinned** in
|
||||
`/etc/apt/preferences.d/intel-gpu-pin` **and** `apt-mark hold`, verified by a simulated
|
||||
upgrade moving 152 packages and touching none of the six.
|
||||
|
||||
## ⚠⚠ Two footguns, both of the silent-wrong class
|
||||
|
||||
**`pct snapshot` REFUSES on a guest with a bind mount AND STILL EXITS 0.** LXC 105 has
|
||||
`mp0: /tank/media`, so Proxmox printed `snapshot feature is not available` and returned
|
||||
`rc=0`. A script trusting that exit code believes it has a backup it does not have. The
|
||||
rootfs is on ZFS, so the working path is `zfs snapshot nvme/subvol-105-disk-0@<tag>` —
|
||||
**and read it back**, per [[feedback_unfalsifiable_at_write_time]].
|
||||
|
||||
**A synthetic `Plex Transcoder` invocation is NOT a valid test of Plex's transcode
|
||||
path, and I burned several rounds proving it.** Plex bundles its own libc among 61
|
||||
libraries; running its ffmpeg from a shell produced three different failure modes
|
||||
(`unknown libva error`, then a libstdc++ `__wmemmove_chk` relocation error) that were
|
||||
artifacts of the harness, and it **failed identically before and after a fix that
|
||||
worked**. No positive control existed, so its negatives carried no information — the
|
||||
exact shape of the tag-detection specimen in the measurement-discipline rule.
|
||||
|
||||
## What actually settles it
|
||||
|
||||
A forced transcode, reading Plex's own log. PASS **names the device**:
|
||||
|
||||
Codecs: testing h264_vaapi (encoder)
|
||||
Codecs: hardware transcoding: testing API vaapi for device '/dev/dri/renderD129' (Intel DG2 [Arc A580])
|
||||
Codecs: testing h264 (decoder) with hwdevice vaapi
|
||||
|
||||
plus `[FFMPEG] - Format 0x… -> bgra` surface enumeration, which only follows a
|
||||
successful `vaInitialize`. FAIL is `final decoder: , final encoder:` with the device
|
||||
never named. ⚠ And an empty-fields line is **also** what a server that never transcoded
|
||||
emits — `TranscodeSession` count was 0, so the original log was an *absence of
|
||||
evidence*, not evidence of failure. I called it failure first and had to withdraw that.
|
||||
|
||||
`vainfo` is a secondary check only (now: iHD 24.3.4, H.264/HEVC VLD **and** EncSliceLP
|
||||
on the Arc) — it exercises the **system** libva, so it can pass while Plex fails.
|
||||
|
||||
Runbook: `docs/runbooks/plex-arc-vaapi-jammy.md`.
|
||||
Rollback: `pct stop 105; zfs rollback nvme/subvol-105-disk-0@pre-vaapi-20260911; pct start 105`.
|
||||
|
||||
**Left alone:** Jellyfin LXC 107 on the same host has the same stale stack and the same
|
||||
Arc available — operator 2026-09-11: not actively used. It ships its own ffmpeg so this
|
||||
fix may not transfer verbatim. LXC 105 also has 152 unrelated pending package upgrades.
|
||||
@@ -1,3 +0,0 @@
|
||||
# `[2026-09-11]` RECOVERY FOOT-GUN, will recur every colo power event: crowdsec crashes on the hard power-off and traefik's bou
|
||||
|
||||
⚠ **RECOVERY FOOT-GUN, will recur every colo power event: crowdsec crashes on the hard power-off and traefik's bouncer fail-CLOSES — empty-body 403 on EVERY HTTP service behind traefik (gitea, homepage, …) while the apps themselves are fine.** Signature (bifrost-dev reported it, gitea-shaped): HTTPS returns `403 content-length 0, no app body` on all routes, but git-over-SSH works (SSH bypasses traefik). Diagnosis: `gitea` direct on `localhost:3000` = 200 (app healthy), through traefik = 403; `crowdsec` container `Exited (255)`; `cscli decisions list` EMPTY (not an IP-ban). **The bouncer plugin does NOT self-recover from a startup-time LAPI-unreachable race** — even after crowdsec is healthy again, traefik keeps 403ing until traefik itself is restarted. **FIX: `docker start crowdsec` (its data/config are LOCAL volumes, comes up clean), wait for `cscli lapi status` = OK, THEN `docker restart traefik`** so the plugin re-inits against the live LAPI. Verified 403→200 on gitea API/web/PyPI-index from an off-box vantage. This unblocked bifrost-dev's 1.2.0 PyPI publish (+ worldtree/wyrd/ratatoskr) and any HTTP gitea access; heid's SSH pushes were never affected. → add to the recovery runbook: **crowdsec+traefik restart is a standard post-power-loss step.**
|
||||
@@ -1,3 +0,0 @@
|
||||
# `[2026-09-11]` Sentinel-R3 pulled, MTP-grafted, and quantized as a M.O.G.-SEC seat candidate — quant DONE, acceptance UNVERIF
|
||||
|
||||
**Sentinel-R3 pulled, MTP-grafted, and quantized as a M.O.G.-SEC seat candidate — quant DONE, acceptance UNVERIFIED (blocked on GPU space).** Operator got access to `glyphsoftware/sentinel-r3` and asked to compare vs the running M.O.G.-SEC seat + pull if promising, then "quant it with a grafted mtp head". **It is promising and a better FIT**: same base (stock Qwen3.8-27B), same `qwen3_5` hybrid arch, same 262K, vision-intact — but M.O.G.-SEC is ALSO an SFT finetune (refusal-free offense+defense cyber SFT — its card: "not a system-prompt sticker on a stock Qwen"; the "persona on stock" wording here is WRONG, corrected 2026-09-14) while Sentinel-R3 is an SFT finetune on 1,230 authorized-pentest agent trajectories over a 19-tool surface that **matches our own harness** (Bash/Read/Write/Edit/Grep/Glob/Agent/Task*/Monitor/…). Card is unusually honest (flags its own mmlu-cybersec 0.88 as within-noise of base). **HF check: M.O.G.-SEC repo unchanged** (sha still our pinned `deede6779…`). **MTP: Sentinel ships ZERO mtp tensors**; grafted the verbatim base head from `qwen38-27b-uncensored-bf16` (compare_mtp_head → IDENTICAL) — lineage correct since Sentinel's base is stock Qwen3.8-27B and that head is a verbatim base graft. ⚠ **Acceptance is UNVERIFIED and may differ from the 47.7% the head hits on STOCK weights** — it now reads hidden states from an SFT-finetuned body (the exact Stage-1b residual risk). Quant = the standard mixed NVFP4-W4A4(MLP 0-55) + FP8-W8A8(attn/linear_attn/lm_head/MLP 56-63) recipe, ran CUDA_VISIBLE_DEVICES=1 on GPU1 free space, no seat downtime, 51→22 GB. post_quant carried the head forward + re-injected `re:^mtp.*` (llm-compressor prunes it → the 0%-accept bug). Structural verify clean: 1968 tensors, 0 unresolved, 15 mtp, 333 visual, ignore has mtp+visual. **Artifact `/tank/aimodels/sentinel-r3-nvfp4-mixed` (+ `.PROVENANCE.txt`).** ⚠ **License is PROPRIETARY** (Glyph Proprietary v1.0, all-rights-reserved) — operator's fair-use/licensee call, not apache like M.O.G.-SEC. ⚠ **Serving/A-B is BLOCKED on GPU space**: weights are 22 GB, GPU0 has 7.6 free / GPU1 19.9 — a probe serve needs a freed co-tenant slot (~25 GB), which is a material-consequence call. Serve with the PROSE system prompt (trained on prose tools, not structured `tools=`). → `/tank/aimodels/sentinel-r3-nvfp4-mixed.PROVENANCE.txt`
|
||||
@@ -1,3 +0,0 @@
|
||||
# `[2026-09-11]` SKALDSONG'S SHAPE SETTLES THE ARCHITECTURE: the adapted completion carrier CANNOT do beat→paragraph, and an in
|
||||
|
||||
⭐⭐⭐ **SKALDSONG'S SHAPE SETTLES THE ARCHITECTURE: the adapted completion carrier CANNOT do beat→paragraph, and an instruct model can. Option C (instruct carrier + corpus rebuilt as instruction→response pairs) is now evidence-backed, not opinion.** Operator's requirement: *"skaldsong will want to write story beats which are a sentence, and have the LLM expound on that sentence to a paragraph and stitch it together."* Booth: `http://10.100.10.50:8090/b/skaldsong-beats/`. **Adapted 4B (checkpoint-75): TEN prompt formats × 3 seeds = 30 samples, ZERO that reliably render the beat** — bare, para-break, labelled, epigraph, fewshot(1), fewshot-bare, fewshot3, elaborate, recount, label-begin. Every one drifts, frames, or truncates. Root cause is structural: *"write a paragraph **about** this sentence"* is an instruction, and a completion model has no mechanism for *about* — it continues the text it is given. ⚠⚠ **Two formats leaked PRETRAINING TASK DATA**: `para-break` emitted an NLI multiple-choice item (*"Does it follow that... OPTIONS: (1). yes (2). it is not possible to tell"*) and `label-begin` a grammar-correction exercise (*"CORRECTION: ... The passage appears to be a sentence fragment"*). A standalone sentence plus a blank line looks exactly like a dataset entry; **style adaptation does not remove base-model task artifacts.** **Instruct arm (`gen` seat + style prompt, no adapter): 10/10 samples inside the requested 90–140 band (124–148w, median 130), every one on-beat, zero drift** — but the voice is generic literary pastiche, abstract-noun-heavy and over-written, not Brontë. **So: voice without direction vs direction without voice; the product needs both.** ⚠ **This applies to Yarros identically** — the carrier question is orthogonal to the author, so the next corpus must NOT re-run this experiment.
|
||||
Reference in New Issue
Block a user