vh
d710e56aca
Split the 53 over-threshold dated log entries (Recent decisions, Tried and abandoned) into per-entry persistent-memory.d/<slug>.md detail files, leaving one-line pointers in the index; the 10 short entries stay inline. Startup index drops 60,527 -> 25,256 bytes (492 -> 270 lines); entry bodies move verbatim to on-demand detail files, so a fresh session loads ~25 KB instead of ~60 KB and pulls a detail file only when its pointer is relevant. Top matter (Repo purpose, Tools & conventions, Current state) is unchanged; both archival back-references preserved. CLAUDE.md persistent-memory section now documents the index<->detail read discipline (read the index, pull details on demand, never bulk-read the dir, commit both together). Auto-archival still held every dated entry back (all <30 days old); the July burst begins aging past the 30-day guard ~2026-07-31.
1.8 KiB
1.8 KiB
[2026-07-08]T1 DPO leg launch — prior BLOCK (now resolved above), kept for the launch recipe. Operator authorized the full DPO stage (via mtf-dev) + went AFK 2h. PROVEN LAUNCH RECIPE (replicates the SFT containeraeon-t1-sftexactly, only--stage sft→dpo):sudo docker run -d --name aeon-t1-dpo --entrypoint python3 --gpus all -e CUDA_VISIBLE_DEVICES=0 -e MTF_FORCE_TRL=1 -e PYTHONPATH=/mtf/src -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -v /home/lkraven/model-training-forge:/mtf -v /tank:/tank -w /mtf aeon-trainer:latest -u scripts/train.py --recipe recipes/training/qwen-3.5-122b-erp-lora/train.aeon-27b.yaml --stage dpo. CRITICAL:--entrypoint python3is REQUIRED — aeon-trainer's default entrypoint is["vllm","serve"](FROM vllm/vllm-openai) → without the override it runs vllm + hits a torch-ABI crash. Dataset verified (pairs_dataset=train.flat.json=1196 pairs). THE BLOCK:from trl import DPOTrainer(backend.py:256) eagerly pulls TRL 0.24.0's WHOLE optional-integration set — cascade: mergekit(missing)→ immutables→mergekit-0.1.4↔pydantic-2.13 HARD incompat (needs pydantic==2.10.6)→llm_blender→dataclasses_json→ llm_blender-0.0.2↔transformers-5.5.0 HARD incompat (TRANSFORMERS_CACHE removed, needs source patch)→weave→ (more). NONE used by our pair-based DPO.pip install mergekitw/deps is UNSAFE (downgrades accelerate 1.14→1.6). Safe partial recipe derived (core libs held: torch2.10/tf5.5.0/trl0.24.0/peft0.19.1/accel1.14.0) but non-convergent → TRULY BLOCKING per operator's carve-out. Did NOT force-hack the proven training image. Handed full diagnosis + recommended fix (lazy-import TRL patch, opt b) to mtf-dev (thread 01KWZG8GJX, expects-reply, monitor armed). gen+rp RESTORED healthy. Relaunch = 1 min once mtf-dev delivers a working image.