vh
9f35c8d659
Six beats through voices-base, lv-bronte, lv-yarros and lv-hemingway, all served from the same process on fv-ml1 :8027 so only the adapter varies. Operator-requested side-by-side. http://10.100.10.50:8090/b/lv-voices-four-arms/ (24h TTL; also on the link board) THE PROMPT NAMES NO AUTHOR, deliberately. Each adapter trained under a prompt naming its own, so driving all four with any one of those hands that arm a hint the others do not get and the page would be measuring the prompt rather than the voice. The shared task skeleton is kept and the author clause removed. One asymmetry is disclosed on the page: Brontë and Hemingway trained on "a SHORT PASSAGE ... may run to several paragraphs" while Yarros trained on "ONE paragraph", so the neutral prompt sits slightly off-distribution for all three rather than for one. THE CONTROL GETS A 4x LARGER TOKEN BUDGET, and publishing it any other way would have been dishonest. Measured at the gate's 320-token budget: voices-base median 26 prose words, 181-257 words of <think> planning first, and 5 of 12 cells never reach the prose at all the adapters 0 of 12 failures each, empty think block in 12 of 12, median 97-105 words The adapters learned to skip the reasoning phase; the carrier has not. Showing the starved control would conflate voice with budget discipline, so the control runs at 1200 tokens and finishes every time, median 121 words. Both numbers are on the page. Two seeds per cell behind a toggle, because one sample of a sampled process is an anecdote, and a blind-mode toggle that hides which column is which. Sampler matches the gate harness (temperature 0.9, top_p 0.95, "BEAT: " prefix). Checked before publishing rather than after: all 36 adapter generations scored for verbatim 8-gram reuse, each arm against ITS OWN corpus. Brontë 0, Yarros 0, Hemingway 2 of 12 with a longest run of 8 words, that run being "i don t know i don t know". Layout verified by rendering it, not by reading the CSS: four equal 374px columns at 1600px wide, no horizontal overflow, 24 cards, 48 panes. ⚠ nh3-dev's shared /opt/ms-playwright tops out at chromium-1234, so playwright must be pinned to 1.61.0; a bare `npm i playwright` pulls 1.63 and asks for a browser build that is not there.
68 lines
3.4 KiB
Python
68 lines
3.4 KiB
Python
"""Four arms, one prompt, one set of beats — harness-matched on the live seat.
|
||
|
||
THE PROMPT IS DELIBERATELY AUTHOR-NEUTRAL. Each adapter was trained under a system
|
||
prompt naming its own author ("in the manner of Charlotte Brontë — mid-nineteenth-century
|
||
first-person retrospective ..."). Driving all four with any ONE of those would hand that
|
||
author's arm a hint the others do not get, and the comparison would measure the prompt.
|
||
So the author clause is removed and the shared task skeleton kept. What is left varies
|
||
only by which LoRA is loaded.
|
||
|
||
⚠ ONE DISCLOSED ASYMMETRY: Brontë and Hemingway trained on "a SHORT PASSAGE ... may run to
|
||
several paragraphs"; Yarros trained on "ONE paragraph". The neutral prompt uses neither
|
||
qualifier, so it sits slightly off-distribution for all three rather than for one.
|
||
|
||
Sampler matches the gate harness exactly: temperature 0.9, top_p 0.95, "BEAT: " prefix.
|
||
Two seeds per cell, because one sample of a sampled process is an anecdote — the second is
|
||
rendered behind a toggle so the page stays readable but the variance is one click away.
|
||
"""
|
||
import json, urllib.request, sys, time
|
||
|
||
SEAT = "http://10.251.50.54:8027/v1/chat/completions"
|
||
ARMS = [("voices-base", "control"), ("lv-bronte", "Brontë"),
|
||
("lv-yarros", "Yarros"), ("lv-hemingway", "Hemingway")]
|
||
SEEDS = [1234, 5678]
|
||
|
||
SYS = ("You expand a single story beat into a SHORT PASSAGE of prose. Render the beat "
|
||
"itself; do not move past it, do not begin a new scene, do not comment, do not "
|
||
"write a chapter heading. Output the prose only, 90–140 words.")
|
||
|
||
BEATS = [
|
||
("b1", "She tells him she is leaving in the morning, and he does not ask her to stay."),
|
||
("b2", "He refuses the money a second time, and the other man sets it down on the table anyway."),
|
||
("b3", "She finds him bleeding on the stairs and asks how long he has been sitting there."),
|
||
("b4", "He waits past the hour they agreed, orders another drink, and watches the door."),
|
||
("b5", "She reads the letter twice, then puts it in the fire without saying what it said."),
|
||
("b6", "He tells her the truth about the accident, and she says nothing for a long time."),
|
||
]
|
||
|
||
|
||
def gen(model, beat, seed):
|
||
body = {"model": model, "seed": seed, "temperature": 0.9, "top_p": 0.95,
|
||
"max_tokens": 320,
|
||
"messages": [{"role": "system", "content": SYS},
|
||
{"role": "user", "content": "BEAT: " + beat}]}
|
||
req = urllib.request.Request(SEAT, data=json.dumps(body).encode(),
|
||
headers={"Content-Type": "application/json"})
|
||
for attempt in range(3):
|
||
try:
|
||
r = json.load(urllib.request.urlopen(req, timeout=180))
|
||
return r["choices"][0]["message"]["content"], r.get("model")
|
||
except Exception as e:
|
||
if attempt == 2:
|
||
raise
|
||
print(f" retry {attempt+1} after {e}", file=sys.stderr)
|
||
time.sleep(5)
|
||
|
||
|
||
out = {"system": SYS, "beats": BEATS, "arms": ARMS, "seeds": SEEDS, "cells": {}}
|
||
t0 = time.time()
|
||
for model, label in ARMS:
|
||
for bid, beat in BEATS:
|
||
for seed in SEEDS:
|
||
txt, served = gen(model, beat, seed)
|
||
out["cells"][f"{model}|{bid}|{seed}"] = {"text": txt, "served": served}
|
||
print(f" {model:<14} {bid} seed={seed} {len(txt.split()):>4}w "
|
||
f"served={served}", flush=True)
|
||
print(f"done in {time.time()-t0:.0f}s")
|
||
json.dump(out, open(sys.argv[1], "w"), ensure_ascii=False, indent=1)
|