Files
esh-pfi-infrastructure/persistent-memory.d/2026-10-12-shared-ci-runner-slow-fsync.md
T

1.3 KiB
Raw Blame History

[2026-10-12] Shared CI runner slow = ospool fsync; fix A (nh3-docker fast runner) done

[2026-10-12] Shared CI runner (ana-docker act_runner, capacity 2, timeout: 30m) is slow because pfi-pve's ospool is HDD mirrors with no SLOG: 8 KiB oflag=dsync writes measured ~145 ms on ana-docker vs ~2.7 ms on nh3-docker (n=3 each; pfi-pve host on ospool 18–32 ms; corviduo-dev ~75 ms). Jobs past 30 min are killed with a truncated log and no footer (albok every run since 10-04; vastblue 10935). Worse since the 10-04 PVE 9 reboot, cause not pinned (drive write cache was the same before and after: sda WCE on, sdb–sdd off). vastblue fixed its side with PostgreSQL on tmpfs (run 11033 green, pytest 14m17s vs 7–8 min per 72-test batch); albok is adding pytest-timeout + durations in 0.1.19. Prime ruled A (2026-10-12 ~1715): an opt-in fast runner on nh3-docker. Done 1725 (nh3-docker-runner, label nh3-docker, no docker socket in jobs, a 4 GiB cap). vastblue's pytest went 14m17s → 4m45s; albok is green (run 11051). ⚠ albok's failures were the slow fsync, never a hang: my 10-12 "hangs" diagnosis was wrong. The ana-docker vCPU-type question (x86-64-v2-AES, no AVX2) is DROPPED: the measured bottleneck was disk fsync plus a hanging network test, not CPU, so nothing rides on it.