servers: add new host dirs, refresh fleet snapshots, orientation doc
Bundles the inventory expansion since 2026-04-22:
- New host dirs (READMEs + ssh-target where dir name doesn't resolve):
ana-nas, ana-wg, esh-vm-db, nh3-nas, pbs-ana, pbs-nh3.
- New PFI VM snapshots (registered + key-installed 2026-04-23):
ana-filebot, pfi-ana-webhost, pfi-postgres, pfi-pteradactyl,
pfi-tacticalrmm, sf-ana-container, sfsrv-ana (system + proxmox).
- servers/irv-ml1: ONBOARDING.md (the first-time setup notes from when
the host was brought into the fleet) + ssh-target (10.100.79.3 over
the WG tunnel — name doesn't DNS-resolve from this workstation).
- servers/{ana-ml2,pfi-pve,sf-r630}/README.md: updates to capture BMC
IPs, the iDRAC vs OS hostname distinction (sf-r630 hardware =
sfsrv-ana OS), and the ana-ml2 Supermicro BMC (10.250.250.50,
distinct from the Dell R750xs iDRAC).
- configs/homepage/docker.yaml: irv-ml1-docker provider added so
homepage auto-discovers irv-ml1's stacks over the WG tunnel.
- docs/orientation.md: narrative fleet overview written for fresh
Claude sessions — sites, backup architecture, governing principles,
gotchas, where-to-look guide. Pointed at from CLAUDE.md.
This commit is contained in:
@@ -0,0 +1,78 @@
|
||||
|
||||
===== HOST =====
|
||||
|
||||
Hostname: ana-filebot
|
||||
Date: 2026-04-23T08:04:59+00:00
|
||||
Uptime: up 34 weeks, 4 days, 7 hours, 50 minutes
|
||||
OS: Debian GNU/Linux 12 (bookworm)
|
||||
Kernel: 6.8.12-8-pve
|
||||
Arch: x86_64
|
||||
|
||||
===== HARDWARE =====
|
||||
|
||||
CPU cores: 4
|
||||
CPU model: Intel(R) Xeon(R) Silver 4310 CPU @ 2.10GHz
|
||||
MemTotal: 2.0 GB
|
||||
MemAvailable: 2.0 GB
|
||||
|
||||
===== GPUS =====
|
||||
|
||||
nvidia-smi not present (no NVIDIA GPUs or driver not installed)
|
||||
|
||||
===== FILESYSTEMS (df) =====
|
||||
|
||||
Filesystem Size Used Avail Use% Mounted on
|
||||
ospool/subvol-112-disk-0 80G 813M 80G 1% /
|
||||
ospool/backup 11T 240M 11T 1% /mnt/ospool-backup
|
||||
NASPool/backupStore 20T 70G 20T 1% /mnt/backup
|
||||
NASPool/db 20T 282M 20T 1% /mnt/db
|
||||
NASPool/docker 20T 384K 20T 1% /mnt/docker
|
||||
NASPool/pve-VMStorage 22T 2.4T 20T 12% /mnt/pve-VMStorage
|
||||
NASPool/webdav 20T 256K 20T 1% /mnt/webdav
|
||||
|
||||
===== PERSISTENT MOUNTS (/etc/fstab, non-comment) =====
|
||||
|
||||
|
||||
===== TARGETED DATA PATHS =====
|
||||
|
||||
/opt (total: 512)
|
||||
total 9
|
||||
drwxr-xr-x 2 root root 2 2023-10-10 13:26 .
|
||||
drwxr-xr-x 17 root root 21 2025-08-24 00:14 ..
|
||||
|
||||
/srv (total: 512)
|
||||
total 9
|
||||
drwxr-xr-x 2 root root 2 2023-10-10 13:26 .
|
||||
drwxr-xr-x 17 root root 21 2025-08-24 00:14 ..
|
||||
|
||||
|
||||
===== DOCKER =====
|
||||
|
||||
docker not installed
|
||||
|
||||
===== COMPOSE FILES (/opt/docker/compose/) =====
|
||||
|
||||
/opt/docker/compose not present
|
||||
|
||||
===== CONFIG LAYOUT (/opt/docker/conf/ — top 200 entries) =====
|
||||
|
||||
/opt/docker/conf not present
|
||||
|
||||
===== LISTENING PORTS =====
|
||||
|
||||
*:22
|
||||
*:8000
|
||||
127.0.0.1:25
|
||||
[::1]:25
|
||||
|
||||
===== MODEL / HUGGINGFACE CACHES =====
|
||||
|
||||
|
||||
===== DOCKER-ADJACENT SYSTEMD SERVICES =====
|
||||
|
||||
container-getty@1.service running
|
||||
container-getty@2.service running
|
||||
|
||||
===== DONE =====
|
||||
|
||||
Paste the above back into the chat, or pass a path as argv[1] to save.
|
||||
@@ -4,11 +4,15 @@ Primary AI inference host for PFI.
|
||||
|
||||
## Network
|
||||
|
||||
- **LAN IP:** 10.250.50.54
|
||||
- **SSH:** standard port 22
|
||||
- **LAN IP:** 10.250.50.54 (in-band, OS-side)
|
||||
- **BMC (OOB):** 10.250.250.50 — Supermicro IPMI web UI
|
||||
at <https://10.250.250.50> (homepage card: *PFI-ANA-ML2 BMC*)
|
||||
- **SSH:** standard port 22 on 10.250.50.54
|
||||
|
||||
## Hardware
|
||||
|
||||
- **Chassis:** Supermicro mid-range inferencing server (bare metal,
|
||||
NOT Dell / not the same box as sf-r630 / sfsrv-ana)
|
||||
- **CPU:** AMD EPYC 9254 24-core (96 threads)
|
||||
- **RAM:** 566 GB
|
||||
- **GPUs:** 2x NVIDIA RTX 6000 Ada Generation (46 GB VRAM each, GPU 0 and GPU 1)
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
# ana-nas
|
||||
|
||||
Debian NFS/SMB file server at the Anaheim site — **CT 109** (LXC) on
|
||||
pfi-pve. Serves the fleet's primary storage layer:
|
||||
|
||||
- `/mnt/backup` → restic target for `rest-server-ana` on ana-docker
|
||||
+ backing storage for PBS-ANA's datastore
|
||||
- `/mnt/docker` → compose/config shares
|
||||
- `/mnt/webdav` → WebDAV data (if still active)
|
||||
- Historically `/mnt/db` (retired 2026-04-23 after the DB-off-NFS
|
||||
migration; pfi-postgres now uses local disk)
|
||||
|
||||
## Network
|
||||
|
||||
- **LAN IP:** 10.250.50.50
|
||||
- **SSH:** `ssh ana-nas` (config alias → `lkraven@10.250.50.50`,
|
||||
key auth). lkraven is a standard user — sudo for root operations.
|
||||
|
||||
## Container context
|
||||
|
||||
This is **CT 109**, not a VM. Bind-mounts into the LXC from pfi-pve's
|
||||
`ospool` ZFS datasets:
|
||||
|
||||
| Inside CT | Host path on pfi-pve | Purpose |
|
||||
|---|---|---|
|
||||
| `/mnt/ospool-backup` | ZFS dataset | ospool snapshots staging |
|
||||
| `/mnt/backup` | ZFS dataset | rest-server-ana + PBS-ANA data |
|
||||
| `/mnt/db` | ZFS dataset | **retired 2026-04-23** (postgres moved to local disk) |
|
||||
| `/mnt/docker` | ZFS dataset | compose files + conf shares |
|
||||
| `/mnt/pve-VMStorage` | ZFS dataset | alt VM storage pool |
|
||||
| `/mnt/webdav` | ZFS dataset | WebDAV data |
|
||||
|
||||
Because these are bind-mounts, CT 109's own vzdump backup (LXC rootfs
|
||||
only) does NOT capture the data payload — the data lives on pfi-pve's
|
||||
ospool and is captured via the host's own backup chain.
|
||||
|
||||
## Why CT 109 matters (SPOF warning)
|
||||
|
||||
As of 2026-04-23, ana-nas still affects these fleet services if it
|
||||
goes offline:
|
||||
|
||||
- **rest-server-ana** (on ana-docker) — its data dir is NFS-mounted
|
||||
from here; an outage kills file-level backups fleet-wide
|
||||
- **PBS-ANA datastore** — NFS-mounted from here; an outage kills
|
||||
VM-level backups fleet-wide
|
||||
- **Cross-site restic rsync source** (ana-nas → nh3-nas daily)
|
||||
|
||||
**Not** affected anymore (as of 2026-04-23 DB migration):
|
||||
|
||||
- pfi-postgres (VM 105) — now on local disk, ana-nas outages
|
||||
don't cascade into vaultwarden/gitea/paperless/zammad
|
||||
|
||||
Full recovery procedure: `memory/incident_ana_nas_spof.md` +
|
||||
`docs/runbooks/disaster-recovery.md`.
|
||||
|
||||
## DSM / rescue access
|
||||
|
||||
Under the hood this is vanilla Debian 12 with hand-configured NFS/SMB.
|
||||
No appliance tooling — share tweaks go through `/etc/exports` +
|
||||
`systemctl reload nfs-server`. Historical confusion about this box
|
||||
being a TrueNAS appliance persists in naming (the Proxmox label was
|
||||
`PFI-ANA-TRUENAS` for a long time) but it is plain Debian.
|
||||
|
||||
## History note
|
||||
|
||||
ID-over-time: this host has been called VM 100, PFI-ANA-TRUENAS,
|
||||
and now CT 109 / ana-nas. Memory file `storage_ana_nas.md` records
|
||||
the VM→CT correction that was clarified 2026-04-23 from the vzdump
|
||||
log output.
|
||||
|
||||
## Refresh
|
||||
|
||||
```
|
||||
scripts/refresh-server-info.sh ana-nas
|
||||
```
|
||||
|
||||
Snapshot at `system-details.txt`.
|
||||
@@ -0,0 +1 @@
|
||||
lkraven@10.250.50.50
|
||||
@@ -0,0 +1,96 @@
|
||||
|
||||
===== HOST =====
|
||||
|
||||
Hostname: ana-nas
|
||||
Date: 2026-04-23T06:53:27+00:00
|
||||
Uptime: up 34 weeks, 3 days, 11 hours, 22 minutes
|
||||
OS: Debian GNU/Linux 12 (bookworm)
|
||||
Kernel: 6.8.12-8-pve
|
||||
Arch: x86_64
|
||||
|
||||
===== HARDWARE =====
|
||||
|
||||
CPU cores: 4
|
||||
CPU model: Intel(R) Xeon(R) Silver 4310 CPU @ 2.10GHz
|
||||
MemTotal: 2.0 GB
|
||||
MemAvailable: 1.9 GB
|
||||
|
||||
===== GPUS =====
|
||||
|
||||
nvidia-smi not present (no NVIDIA GPUs or driver not installed)
|
||||
|
||||
===== FILESYSTEMS (df) =====
|
||||
|
||||
Filesystem Size Used Avail Use% Mounted on
|
||||
ospool/subvol-109-disk-0 80G 813M 80G 1% /
|
||||
ospool/backup 11T 240M 11T 1% /mnt/ospool-backup
|
||||
NASPool/backupStore 20T 67G 20T 1% /mnt/backup
|
||||
NASPool/db 20T 282M 20T 1% /mnt/db
|
||||
NASPool/docker 20T 384K 20T 1% /mnt/docker
|
||||
NASPool/pve-VMStorage 22T 2.4T 20T 12% /mnt/pve-VMStorage
|
||||
NASPool/webdav 20T 256K 20T 1% /mnt/webdav
|
||||
|
||||
===== PERSISTENT MOUNTS (/etc/fstab, non-comment) =====
|
||||
|
||||
|
||||
===== TARGETED DATA PATHS =====
|
||||
|
||||
/opt (total: 512)
|
||||
total 9
|
||||
drwxr-xr-x 2 root root 2 2023-10-10 13:26 .
|
||||
drwxr-xr-x 17 root root 21 2025-08-24 19:31 ..
|
||||
|
||||
/srv (total: 512)
|
||||
total 9
|
||||
drwxr-xr-x 2 root root 2 2023-10-10 13:26 .
|
||||
drwxr-xr-x 17 root root 21 2025-08-24 19:31 ..
|
||||
|
||||
|
||||
===== DOCKER =====
|
||||
|
||||
docker not installed
|
||||
|
||||
===== COMPOSE FILES (/opt/docker/compose/) =====
|
||||
|
||||
/opt/docker/compose not present
|
||||
|
||||
===== CONFIG LAYOUT (/opt/docker/conf/ — top 200 entries) =====
|
||||
|
||||
/opt/docker/conf not present
|
||||
|
||||
===== LISTENING PORTS =====
|
||||
|
||||
*:22
|
||||
*:9090
|
||||
0.0.0.0:111
|
||||
0.0.0.0:139
|
||||
0.0.0.0:2049
|
||||
0.0.0.0:39347
|
||||
0.0.0.0:40609
|
||||
0.0.0.0:445
|
||||
0.0.0.0:48585
|
||||
0.0.0.0:49363
|
||||
0.0.0.0:57311
|
||||
127.0.0.1:25
|
||||
[::1]:25
|
||||
[::]:111
|
||||
[::]:139
|
||||
[::]:2049
|
||||
[::]:41319
|
||||
[::]:445
|
||||
[::]:47023
|
||||
[::]:50729
|
||||
[::]:50951
|
||||
[::]:55887
|
||||
|
||||
===== MODEL / HUGGINGFACE CACHES =====
|
||||
|
||||
|
||||
===== DOCKER-ADJACENT SYSTEMD SERVICES =====
|
||||
|
||||
container-getty@1.service running
|
||||
container-getty@2.service running
|
||||
|
||||
===== DONE =====
|
||||
|
||||
Paste the above back into the chat, or pass a path as argv[1] to save.
|
||||
@@ -0,0 +1,73 @@
|
||||
|
||||
===== HOST =====
|
||||
|
||||
Hostname: ana-wg
|
||||
Date: 2026-04-23T08:05:00+00:00
|
||||
Uptime: up 34 weeks, 4 days, 7 hours, 50 minutes
|
||||
OS: Debian GNU/Linux 12 (bookworm)
|
||||
Kernel: 6.8.12-8-pve
|
||||
Arch: x86_64
|
||||
|
||||
===== HARDWARE =====
|
||||
|
||||
CPU cores: 4
|
||||
CPU model: Intel(R) Xeon(R) Silver 4310 CPU @ 2.10GHz
|
||||
MemTotal: 2.0 GB
|
||||
MemAvailable: 2.0 GB
|
||||
|
||||
===== GPUS =====
|
||||
|
||||
nvidia-smi not present (no NVIDIA GPUs or driver not installed)
|
||||
|
||||
===== FILESYSTEMS (df) =====
|
||||
|
||||
Filesystem Size Used Avail Use% Mounted on
|
||||
ospool/subvol-113-disk-0 8.0G 623M 7.4G 8% /
|
||||
|
||||
===== PERSISTENT MOUNTS (/etc/fstab, non-comment) =====
|
||||
|
||||
|
||||
===== TARGETED DATA PATHS =====
|
||||
|
||||
/opt (total: 1.0K)
|
||||
total 10
|
||||
drwxr-xr-x 2 root root 3 2024-10-30 18:08 .
|
||||
drwxr-xr-x 17 root root 21 2025-08-24 00:14 ..
|
||||
lrwxrwxrwx 1 root root 29 2024-10-30 18:08 pivpn -> /usr/local/src/pivpn/scripts/
|
||||
|
||||
/srv (total: 512)
|
||||
total 9
|
||||
drwxr-xr-x 2 root root 2 2023-10-10 13:26 .
|
||||
drwxr-xr-x 17 root root 21 2025-08-24 00:14 ..
|
||||
|
||||
|
||||
===== DOCKER =====
|
||||
|
||||
docker not installed
|
||||
|
||||
===== COMPOSE FILES (/opt/docker/compose/) =====
|
||||
|
||||
/opt/docker/compose not present
|
||||
|
||||
===== CONFIG LAYOUT (/opt/docker/conf/ — top 200 entries) =====
|
||||
|
||||
/opt/docker/conf not present
|
||||
|
||||
===== LISTENING PORTS =====
|
||||
|
||||
0.0.0.0:22
|
||||
127.0.0.1:25
|
||||
[::1]:25
|
||||
[::]:22
|
||||
|
||||
===== MODEL / HUGGINGFACE CACHES =====
|
||||
|
||||
|
||||
===== DOCKER-ADJACENT SYSTEMD SERVICES =====
|
||||
|
||||
container-getty@1.service running
|
||||
container-getty@2.service running
|
||||
|
||||
===== DONE =====
|
||||
|
||||
Paste the above back into the chat, or pass a path as argv[1] to save.
|
||||
@@ -0,0 +1,96 @@
|
||||
# esh-vm-db
|
||||
|
||||
ESH-side database VM at `esteban.net`. Hosts **two database servers**:
|
||||
|
||||
- **PostgreSQL 15** on port 5432 — primary consumer is paperless-ngx
|
||||
on esh-docker-vm (`paperless-ng` user/db).
|
||||
- **MongoDB** on port 27017 — consumers unknown; document at next
|
||||
inventory pass.
|
||||
|
||||
Discovered 2026-04-23 during the paperless-ngx password rotation —
|
||||
paperless's `PAPERLESS_DBHOST: 10.0.50.60` pointed here rather than
|
||||
the assumed pfi-postgres (VM 105). Up until that point this host was
|
||||
invisible to inventory.
|
||||
|
||||
## Network
|
||||
|
||||
- **LAN IP:** 10.0.50.60
|
||||
- **FQDN:** `esh-vm-db.esteban.net`
|
||||
- **SSH:** `ssh esh-vm-db` (config alias → `lkraven@10.0.50.60`,
|
||||
key auth).
|
||||
- **Listening ports:** 22 (ssh), 111 (rpcbind — NFS mount support),
|
||||
5432 (postgres), 27017 (mongo).
|
||||
|
||||
## Hardware
|
||||
|
||||
- **Hypervisor:** presumably esh-pve or esh-pve-nas (to be confirmed
|
||||
— not yet queried from the PVE side).
|
||||
- **CPU:** Intel Core i9-13900H (16 cores, mobile) — indicates this
|
||||
VM lives on a mini-PC / NUC-class ESH host.
|
||||
- **RAM:** 7.8 GB
|
||||
- **Root disk:** 250 GB ext4 on `/dev/sda1`, 5.1 GB used (2%). Plenty
|
||||
of headroom for both DBs.
|
||||
- **OS:** Debian 12 (bookworm), kernel 6.1.0-23
|
||||
- **Uptime:** 3+ weeks at last snapshot.
|
||||
|
||||
## What it runs
|
||||
|
||||
- **PostgreSQL 15 (apt package, `postgresql`)** — data directory
|
||||
default (`/var/lib/postgresql/15/main` presumably — confirm with
|
||||
`pg_lsclusters` on a future refresh). Local disk, not NFS —
|
||||
consistent with the "DBs must live on local disk" principle we
|
||||
adopted 2026-04-23.
|
||||
- **MongoDB** — port 27017 bound to 0.0.0.0. DataPath + version to
|
||||
be captured.
|
||||
- **No Docker.** `server_inspect.sh` reports `docker not installed`
|
||||
and `/opt/docker` absent — this host is bare-metal DB service,
|
||||
not a container host.
|
||||
|
||||
## Storage
|
||||
|
||||
- `/dev/sda1` (250 GB ext4) — DB data + OS
|
||||
- `/mnt/backup` mounted from `10.0.50.50:/mnt/backup` via NFS
|
||||
(`defaults`). Purpose TBD — possibly for pg_dump outputs or other
|
||||
backup staging. Review on next pass.
|
||||
|
||||
## Consumers
|
||||
|
||||
- **paperless-ngx** on esh-docker-vm — uses postgres user
|
||||
`paperless-ng` against DB `paperless-ng` (confirmed 2026-04-23,
|
||||
password rotated).
|
||||
- **MongoDB consumers** — unknown; map these by checking connection
|
||||
logs or by inventorying the other ESH-side Docker stacks for
|
||||
`27017`/`mongodb` references.
|
||||
|
||||
## Backup coverage
|
||||
|
||||
**Not yet configured.** Same category as ana-ml2/irv-ml1/esh-vm-db
|
||||
now (bare-metal DB host without vzdump awareness from the PVE side).
|
||||
|
||||
Next steps:
|
||||
|
||||
- Add to fleet PBS vzdump schedule (its hypervisor backs it up via
|
||||
PBS-ANA if the hypervisor is onboarded).
|
||||
- Draft `configs/restic/esh-vm-db/profiles.yaml` for file-level +
|
||||
per-DB dumps (`pg_dumpall` for postgres, `mongodump` for mongo).
|
||||
Target: `rest-server-ana` via the ESH → ANA cross-WAN path, same
|
||||
as other ESH-side restic clients.
|
||||
- `rest-server-ana`'s `.htpasswd` will need an `esh-vm-db` entry.
|
||||
|
||||
## Refresh state
|
||||
|
||||
```bash
|
||||
scripts/refresh-server-info.sh esh-vm-db
|
||||
```
|
||||
|
||||
## Follow-ups (do soon)
|
||||
|
||||
- Confirm hypervisor (likely esh-pve or esh-pve-nas) — `qm list`
|
||||
on each and find VM with this name.
|
||||
- Map MongoDB consumers.
|
||||
- Configure backup (restic profile + per-DB dumps + vzdump coverage
|
||||
on the hypervisor side).
|
||||
- Audit postgres `pg_hba.conf` — is 5432 bound to 0.0.0.0 with
|
||||
proper host-based auth, or is it over-exposed? paperless connects
|
||||
from 10.0.50.45 (esh-docker-vm); other hosts shouldn't be able
|
||||
to reach in.
|
||||
@@ -0,0 +1 @@
|
||||
lkraven@10.0.50.60
|
||||
@@ -0,0 +1,80 @@
|
||||
|
||||
===== HOST =====
|
||||
|
||||
Hostname: esh-vm-db.esteban.net
|
||||
Date: 2026-04-23T21:53:16-07:00
|
||||
Uptime: up 3 weeks, 2 days, 23 hours, 22 minutes
|
||||
OS: Debian GNU/Linux 12 (bookworm)
|
||||
Kernel: 6.1.0-23-amd64
|
||||
Arch: x86_64
|
||||
|
||||
===== HARDWARE =====
|
||||
|
||||
CPU cores: 16
|
||||
CPU model: 13th Gen Intel(R) Core(TM) i9-13900H
|
||||
MemTotal: 7.8 GB
|
||||
MemAvailable: 7.2 GB
|
||||
|
||||
===== GPUS =====
|
||||
|
||||
nvidia-smi not present (no NVIDIA GPUs or driver not installed)
|
||||
|
||||
===== FILESYSTEMS (df) =====
|
||||
|
||||
Filesystem Size Used Avail Use% Mounted on
|
||||
/dev/sda1 250G 5.1G 233G 3% /
|
||||
10.0.50.50:/mnt/backup 92T 11G 92T 1% /mnt/backup
|
||||
|
||||
===== PERSISTENT MOUNTS (/etc/fstab, non-comment) =====
|
||||
|
||||
UUID=9022f0b6-4299-452a-a4a4-85e1623243ae / ext4 errors=remount-ro 0 1
|
||||
UUID=6d90bc64-6f3e-41b9-8295-640ac854e5e5 none swap sw 0 0
|
||||
/dev/sr0 /media/cdrom0 udf,iso9660 user,noauto 0 0
|
||||
10.0.50.50:/mnt/backup /mnt/backup nfs defaults 0 0
|
||||
|
||||
===== TARGETED DATA PATHS =====
|
||||
|
||||
/opt (total: 4.0K)
|
||||
total 8
|
||||
drwxr-xr-x 2 root root 4096 2024-05-20 23:47 .
|
||||
drwxr-xr-x 18 root root 4096 2024-08-13 17:35 ..
|
||||
|
||||
/srv (total: 2.8M)
|
||||
total 12
|
||||
drwxr-xr-x 3 root root 4096 2024-06-13 10:05 .
|
||||
drwxr-xr-x 18 root root 4096 2024-08-13 17:35 ..
|
||||
drwxrwxrwx 4 root root 4096 2024-06-13 10:44 backups
|
||||
|
||||
|
||||
===== DOCKER =====
|
||||
|
||||
docker not installed
|
||||
|
||||
===== COMPOSE FILES (/opt/docker/compose/) =====
|
||||
|
||||
/opt/docker/compose not present
|
||||
|
||||
===== CONFIG LAYOUT (/opt/docker/conf/ — top 200 entries) =====
|
||||
|
||||
/opt/docker/conf not present
|
||||
|
||||
===== LISTENING PORTS =====
|
||||
|
||||
0.0.0.0:111
|
||||
0.0.0.0:22
|
||||
0.0.0.0:27017
|
||||
0.0.0.0:5432
|
||||
[::]:111
|
||||
[::]:22
|
||||
[::]:5432
|
||||
|
||||
===== MODEL / HUGGINGFACE CACHES =====
|
||||
|
||||
|
||||
===== DOCKER-ADJACENT SYSTEMD SERVICES =====
|
||||
|
||||
(none matching)
|
||||
|
||||
===== DONE =====
|
||||
|
||||
Paste the above back into the chat, or pass a path as argv[1] to save.
|
||||
@@ -0,0 +1,144 @@
|
||||
# irv-ml1 onboarding checklist
|
||||
|
||||
Steps to bring irv-ml1 fully into the fleet, matching the conventions
|
||||
used by the other hosts.
|
||||
|
||||
## Already done (repo-side)
|
||||
|
||||
- [x] `servers/irv-ml1/` registered, ssh-target + system-details captured
|
||||
- [x] SSH config alias `ssh irv-ml1`
|
||||
- [x] CLAUDE.md Servers table + Placement rules updated
|
||||
- [x] `configs/restic/irv-ml1/profiles.yaml` + README drafted
|
||||
- [x] Homepage card under new `Infra - IRV` group (IRV-ML1 ping monitor)
|
||||
- [x] Homepage `docker.yaml` entry staged (commented) for Docker auto-discovery
|
||||
- [x] Homepage `settings.yaml` — `Infra - IRV` group added to Infrastructure tab
|
||||
|
||||
## To do on the host
|
||||
|
||||
### 1. Rename hostname (optional, non-blocking)
|
||||
|
||||
```bash
|
||||
ssh -t irv-ml1 'sudo hostnamectl set-hostname irv-ml1 && \
|
||||
sudo sed -i "s/ana-ml1/irv-ml1/g" /etc/hosts && \
|
||||
hostname && cat /etc/hostname'
|
||||
scripts/refresh-server-info.sh irv-ml1 # capture new identity
|
||||
```
|
||||
|
||||
### 2. Add lkraven to docker group (blocks server_inspect full output)
|
||||
|
||||
```bash
|
||||
ssh -t irv-ml1 'sudo usermod -aG docker lkraven'
|
||||
# then logout + login for group to take effect
|
||||
```
|
||||
|
||||
### 3. Install Docker + Dockge (your call on convention adoption)
|
||||
|
||||
Once installed, create the fleet-standard directory layout:
|
||||
|
||||
```bash
|
||||
ssh -t irv-ml1 'sudo mkdir -p /opt/docker/{compose,conf} && \
|
||||
sudo chown -R lkraven:lkraven /opt/docker'
|
||||
```
|
||||
|
||||
Create the external traefik-net (even if no traefik here yet — stacks
|
||||
reference it as `external: true`):
|
||||
|
||||
```bash
|
||||
ssh -t irv-ml1 'docker network create traefik-net'
|
||||
```
|
||||
|
||||
### 4. Expose Docker socket on port 2375 (for homepage auto-discovery)
|
||||
|
||||
On irv-ml1:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /etc/systemd/system/docker.service.d
|
||||
sudo tee /etc/systemd/system/docker.service.d/override.conf > /dev/null <<'EOF'
|
||||
[Service]
|
||||
ExecStart=
|
||||
ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375
|
||||
EOF
|
||||
sudo systemctl daemon-reload && sudo systemctl restart docker
|
||||
```
|
||||
|
||||
Then uncomment the `irv-ml1-docker:` block in
|
||||
`configs/homepage/docker.yaml` and push homepage config.
|
||||
|
||||
**Trust model:** 2375 is unencrypted; only safe because it's bound to
|
||||
the WG tunnel interface on the remote side. Confirm your firewall
|
||||
rules prevent exposure of 2375 on any non-WG interface.
|
||||
|
||||
### 5. Deploy Dockge
|
||||
|
||||
Standard fleet pattern — see `stacks/dockge/` or copy from
|
||||
esh-docker-vm/nh3-docker. Port 5001. After Dockge is up:
|
||||
|
||||
- Uncomment the `IRV-ML1-Dockge` entry in `configs/homepage/services.yaml`
|
||||
- Push homepage config
|
||||
|
||||
### 6. Deploy monitoring agents
|
||||
|
||||
Cross-site Beszel + Dozzle agents report to hubs on ana-docker:
|
||||
|
||||
- `beszel-agent` → ana-docker's Beszel hub
|
||||
- `dozzle-agent` → ana-docker's Dozzle hub
|
||||
|
||||
Copy compose files from another host (`stacks/beszel-agent/`,
|
||||
`stacks/dozzle-agent/`) and adjust `HUB_URL` / TOKEN / display name
|
||||
for irv-ml1.
|
||||
|
||||
### 7. Set up restic
|
||||
|
||||
Follow `configs/restic/irv-ml1/README.md` exactly:
|
||||
|
||||
1. Create `irv-ml1` htpasswd entry on rest-server-nh3 (not rest-server-ana
|
||||
— we target the NH3 side because it's the WG termination endpoint)
|
||||
2. Install `/etc/restic/restic.env` + `/etc/restic/password` on irv-ml1
|
||||
3. `restic init`
|
||||
4. Install resticprofile
|
||||
5. Deploy `profiles.yaml`
|
||||
6. `resticprofile schedule --all`
|
||||
7. First manual backup
|
||||
8. Verify snapshot appears in Backrest UI on ana-docker
|
||||
|
||||
### 8. Push homepage config to make cards live
|
||||
|
||||
After each homepage YAML edit in this repo:
|
||||
|
||||
```bash
|
||||
rsync -av --exclude='.env' --exclude='.env.*' \
|
||||
configs/homepage/ esh-docker-vm:/opt/docker/conf/homepage/
|
||||
```
|
||||
|
||||
## Verification after everything
|
||||
|
||||
```bash
|
||||
# Host reachable through WG
|
||||
ssh irv-ml1 hostname
|
||||
|
||||
# Docker up, 2375 listening
|
||||
ssh irv-ml1 'docker ps && ss -tlnp | grep 2375'
|
||||
|
||||
# Dockge up
|
||||
curl -fsS http://10.100.79.3:5001/ >/dev/null && echo DOCKGE_OK
|
||||
|
||||
# Agents reporting (check from ana-docker)
|
||||
ssh ana-docker 'docker exec beszel docker logs 2>&1 | grep irv-ml1 | tail'
|
||||
|
||||
# Restic timer armed
|
||||
ssh irv-ml1 'systemctl list-timers "resticprofile*"'
|
||||
|
||||
# First restic snapshot visible
|
||||
# -> Backrest UI → repos → irv-ml1
|
||||
```
|
||||
|
||||
## WireGuard health considerations
|
||||
|
||||
Everything this host does depends on the WG tunnel. Worth adding:
|
||||
|
||||
- Beszel alert on ping loss to 10.100.50.50 (WG peer)
|
||||
- Or a simple cron on ana-docker that pings 10.100.79.3 every 5 min
|
||||
and alerts via mailrise on failure
|
||||
|
||||
Not blocking onboarding, but the tunnel is the hidden SPOF for every
|
||||
irv-ml1 service.
|
||||
@@ -0,0 +1 @@
|
||||
lkraven@10.100.79.3
|
||||
@@ -0,0 +1,75 @@
|
||||
# nh3-nas
|
||||
|
||||
Synology RS2418+ at the NH3 site (`PFI-NH3-NAS`, `10.100.50.50`).
|
||||
Primary role is fleet storage: VM-image hosting for nh3-pve, the
|
||||
NH3-side restic target, and as of 2026-04-22 the PBS-NH3 datastore
|
||||
backend.
|
||||
|
||||
## Network
|
||||
|
||||
- **LAN IP:** 10.100.50.50
|
||||
- **DSM web UI:** <https://10.100.50.50:5001>
|
||||
- **SSH:** `ssh nh3-nas` (config alias → `syncuser@10.100.50.50`,
|
||||
key auth). The `syncuser` account was created specifically for
|
||||
automation/tooling access; `admin` retained as the DSM primary.
|
||||
|
||||
## Hardware
|
||||
|
||||
- **Model:** Synology RS2418+ (denverton platform)
|
||||
- **CPU:** Intel Atom C3538 @ 2.10 GHz, 4 cores
|
||||
- **RAM:** 31.3 GB
|
||||
- **Kernel:** `4.4.302+` — DSM's custom kernel, not a vanilla Debian
|
||||
base. Implications: very old bash features only (4.x), BusyBox-ish
|
||||
userland for some tools, syno_acl layer on Btrfs.
|
||||
- **Volume:** `/volume1`, Btrfs on `cachedev_0` (SSD-cached LVM).
|
||||
42 TB total, 27 TB used (64%), 16 TB free.
|
||||
|
||||
## Services running on the box
|
||||
|
||||
- **DSM built-in NFS server** — exports under `/etc/exports` managed
|
||||
via DSM Shared Folder UI. Active exports include:
|
||||
- `/volume1/Shared`, `/volume1/Media`, `/volume1/NetBackup`,
|
||||
`/volume1/Backup`, `/volume1/compose`, `/volume1/VMStorage`,
|
||||
`/volume1/devstor` (legacy home-lab exports; `all_squash`)
|
||||
- `/volume1/pbs` — dedicated share for PBS-NH3's datastore mount.
|
||||
Linux/POSIX mode 777, no syno_acl, `no_root_squash + no_all_squash`.
|
||||
Do NOT enable Advanced Permissions on this share — see
|
||||
`docs/runbooks/pbs-deployment.md` Phase 5.3 for the history.
|
||||
- **rest-server-nh3** (Docker via DSM ContainerManager) — restic
|
||||
HTTP endpoint on port 8000, writes under `/volume1/Backup`. Serves
|
||||
nh3-docker and nh3-dev restic clients.
|
||||
|
||||
## What backs up to it
|
||||
|
||||
- nh3-docker + nh3-dev resticprofile timers (via rest-server-nh3)
|
||||
- PBS-NH3 datastore (`backups-mirror`), which receives the nightly
|
||||
sync pull from PBS-ANA
|
||||
|
||||
## What backs up FROM it
|
||||
|
||||
Not currently backed up itself — the DSM side is the source-of-truth
|
||||
for its own config. Future work: cross-site rsync of
|
||||
`/volume1/Backup/restic/...` to the Ana NAS (`10.250.50.50`), blocked
|
||||
previously on SSH-access to this host (now resolved with `syncuser`).
|
||||
|
||||
## DSM-specific gotchas
|
||||
|
||||
- **Docker:** runs via DSM's ContainerManager package, not a native
|
||||
`docker` CLI. `docker ps` over SSH returns empty; use
|
||||
`sudo synopkg list | grep -i container` and
|
||||
`sudo docker ...` (DSM aliases the binary into root's PATH).
|
||||
- **Home directories:** live under `/var/services/homes/<user>/` (not
|
||||
`/home/<user>`). SSH public keys go in `~/.ssh/authorized_keys`
|
||||
which maps to the DSM home. Home dir permission must be 755 (not
|
||||
700) or sshd rejects keys silently after DSM updates reset it.
|
||||
- **Btrfs mount options** include `synoacl` — POSIX permissions are
|
||||
projected through Synology's ACL layer. For share permissions to
|
||||
behave as plain POSIX, either turn off "Advanced Permissions" in
|
||||
DSM *and* flatten with `chmod 777` (which converts the share to
|
||||
"Linux mode", `synoacltool -get` will confirm), or grant explicit
|
||||
ACL entries with `synoacltool -add`.
|
||||
|
||||
## Refresh state
|
||||
|
||||
Snapshot captured via `scripts/refresh-server-info.sh nh3-nas` —
|
||||
updates `system-details.txt` alongside this README.
|
||||
@@ -0,0 +1 @@
|
||||
syncuser@10.100.50.50
|
||||
@@ -0,0 +1,126 @@
|
||||
|
||||
===== HOST =====
|
||||
|
||||
Hostname: PFI-NH3-NAS
|
||||
Date: 2026-04-22T23:46:21-07:00
|
||||
Uptime: up 3 days, 9 hours, 26 minutes
|
||||
Kernel: 4.4.302+
|
||||
Arch: x86_64
|
||||
|
||||
===== HARDWARE =====
|
||||
|
||||
CPU cores: 4
|
||||
CPU model: Intel(R) Atom(TM) CPU C3538 @ 2.10GHz
|
||||
MemTotal: 31.3 GB
|
||||
MemAvailable: 29.5 GB
|
||||
|
||||
===== GPUS =====
|
||||
|
||||
nvidia-smi not present (no NVIDIA GPUs or driver not installed)
|
||||
|
||||
===== FILESYSTEMS (df) =====
|
||||
|
||||
Filesystem Size Used Avail Use% Mounted on
|
||||
/dev/md0 2.3G 1.5G 749M 67% /
|
||||
/dev/mapper/cachedev_0 42T 27T 16T 64% /volume1
|
||||
/dev/loop0 27M 767K 24M 4% /tmp/SynologyAuthService
|
||||
/dev/mapper/cachedev_0 42T 27T 16T 64% /volume1/@appdata/ContainerManager/all_shares/Backup
|
||||
/dev/mapper/cachedev_0 42T 27T 16T 64% /volume1/@appdata/ContainerManager/all_shares/compose
|
||||
/dev/mapper/cachedev_0 42T 27T 16T 64% /volume1/@appdata/ContainerManager/all_shares/devstor
|
||||
/dev/mapper/cachedev_0 42T 27T 16T 64% /volume1/@appdata/ContainerManager/all_shares/docker
|
||||
/dev/mapper/cachedev_0 42T 27T 16T 64% /volume1/@appdata/ContainerManager/all_shares/homes
|
||||
/dev/mapper/cachedev_0 42T 27T 16T 64% /volume1/@appdata/ContainerManager/all_shares/Media
|
||||
/dev/mapper/cachedev_0 42T 27T 16T 64% /volume1/@appdata/ContainerManager/all_shares/Music
|
||||
/dev/mapper/cachedev_0 42T 27T 16T 64% /volume1/@appdata/ContainerManager/all_shares/NetBackup
|
||||
/dev/mapper/cachedev_0 42T 27T 16T 64% /volume1/@appdata/ContainerManager/all_shares/Shared
|
||||
/dev/mapper/cachedev_0 42T 27T 16T 64% /volume1/@appdata/ContainerManager/all_shares/VMStorage
|
||||
/dev/mapper/cachedev_0 42T 27T 16T 64% /volume1/@appdata/ContainerManager/all_shares/web
|
||||
/dev/mapper/cachedev_0 42T 27T 16T 64% /volume1/@appdata/ContainerManager/all_shares/web_packages
|
||||
/dev/mapper/cachedev_0 42T 27T 16T 64% /volume1/@appdata/ContainerManager/all_shares/pbs
|
||||
|
||||
===== PERSISTENT MOUNTS (/etc/fstab, non-comment) =====
|
||||
|
||||
none /proc proc defaults 0 0
|
||||
/dev/root / ext4 defaults 1 1
|
||||
/dev/mapper/cachedev_0 /volume1 btrfs auto_reclaim_space,ssd,synoacl,relatime,nodev 0 0
|
||||
|
||||
===== TARGETED DATA PATHS =====
|
||||
|
||||
/opt (total: 4.0K)
|
||||
|
||||
|
||||
===== DOCKER =====
|
||||
|
||||
docker not installed
|
||||
|
||||
===== COMPOSE FILES (/opt/docker/compose/) =====
|
||||
|
||||
/opt/docker/compose not present
|
||||
|
||||
===== CONFIG LAYOUT (/opt/docker/conf/ — top 200 entries) =====
|
||||
|
||||
/opt/docker/conf not present
|
||||
|
||||
===== LISTENING PORTS =====
|
||||
|
||||
0.0.0.0:111
|
||||
0.0.0.0:139
|
||||
0.0.0.0:161
|
||||
0.0.0.0:2022
|
||||
0.0.0.0:2049
|
||||
0.0.0.0:22
|
||||
0.0.0.0:4045
|
||||
0.0.0.0:443
|
||||
0.0.0.0:445
|
||||
0.0.0.0:5000
|
||||
0.0.0.0:5001
|
||||
0.0.0.0:5357
|
||||
0.0.0.0:662
|
||||
0.0.0.0:80
|
||||
0.0.0.0:8000
|
||||
0.0.0.0:873
|
||||
0.0.0.0:892
|
||||
10.100.250.50:3260
|
||||
10.100.50.50:3260
|
||||
127.0.0.1:33304
|
||||
127.0.0.1:512
|
||||
127.0.0.1:5432
|
||||
:::111
|
||||
:::139
|
||||
:::161
|
||||
:::2022
|
||||
:::2049
|
||||
:::22
|
||||
:::22000
|
||||
:::3261
|
||||
:::3263
|
||||
:::3264
|
||||
:::3265
|
||||
:::4045
|
||||
:::443
|
||||
:::445
|
||||
:::5000
|
||||
:::5001
|
||||
:::5357
|
||||
:::6281
|
||||
:::662
|
||||
:::80
|
||||
:::8000
|
||||
:::8384
|
||||
:::873
|
||||
:::892
|
||||
fe80::211:32ff:fe9:3260
|
||||
fe80::ee0d:9aff:fe:3260
|
||||
|
||||
===== MODEL / HUGGINGFACE CACHES =====
|
||||
|
||||
|
||||
===== DOCKER-ADJACENT SYSTEMD SERVICES =====
|
||||
|
||||
pkg-ContainerManager-dockerd.service running
|
||||
pkg-ContainerManager-event-watcherd.service running
|
||||
pkg-ContainerManager-termd.service running
|
||||
|
||||
===== DONE =====
|
||||
|
||||
Paste the above back into the chat, or pass a path as argv[1] to save.
|
||||
@@ -0,0 +1,87 @@
|
||||
# pbs-ana
|
||||
|
||||
Proxmox Backup Server — **fleet primary**. VM on pfi-pve with an
|
||||
NFS-mounted datastore on the Ana NAS. All 5 hypervisors (pfi-pve,
|
||||
nh3-pve, esh-pve, esh-pve-nas, sfsrv-ana) back up here; one-way sync
|
||||
to pbs-nh3 at 06:00 daily.
|
||||
|
||||
## Network
|
||||
|
||||
- **LAN IP:** 10.250.50.90
|
||||
- **FQDN:** `pbs-ana.phasefinal.com`
|
||||
- **PBS web UI:** <https://10.250.50.90:8007/>
|
||||
- **SSH:** `ssh pbs-ana` (config alias → `lkraven@10.250.50.90`).
|
||||
Root ops via sudo from lkraven.
|
||||
- **Homepage card:** *PBS-ANA* in `Infra - ANA` group.
|
||||
|
||||
## Hardware (VM)
|
||||
|
||||
- **Hypervisor:** pfi-pve
|
||||
- **vCPU:** 4 (QEMU)
|
||||
- **RAM:** 7.8 GB
|
||||
- **Root disk:** `/dev/sda1` 31 GB ext4 — OS only (2 GB used)
|
||||
- **Datastore:** **NFS from ana-nas** at `/mnt/pbs-datastore` →
|
||||
`10.250.50.50:/mnt/backup/pbs-ana`, 20 TB (currently 336 GB used)
|
||||
|
||||
## NFS details (the ZFS-case-insensitivity gotcha)
|
||||
|
||||
The ZFS dataset backing `/mnt/backup` on ana-nas is
|
||||
`casesensitivity=insensitive`, which breaks NFSv4 writes (EACCES).
|
||||
Mount must be **NFSv3** — that's what `/etc/fstab` already uses:
|
||||
|
||||
```
|
||||
10.250.50.50:/mnt/backup/pbs-ana /mnt/pbs-datastore nfs \
|
||||
defaults,_netdev,bg,hard,timeo=600,retrans=2,vers=3 0 0
|
||||
```
|
||||
|
||||
Don't upgrade this to vers=4 without fixing the dataset. See
|
||||
`docs/runbooks/pbs-deployment.md` Phase 0.5 for the full history.
|
||||
|
||||
## What runs here
|
||||
|
||||
- `proxmox-backup-proxy` — HTTPS API + web UI (port 8007)
|
||||
- `proxmox-backup` — backend service
|
||||
- Scheduled verify job: weekly Saturday 23:00 (`ignore-verified=true`,
|
||||
`outdated-after=30 days`)
|
||||
- Garbage collection: default weekly
|
||||
|
||||
## Namespaces
|
||||
|
||||
Per-hypervisor namespaces to prevent VMID collisions across PVE
|
||||
hosts (VMIDs are NOT globally unique):
|
||||
|
||||
- `pfi-pve`
|
||||
- `nh3-pve`
|
||||
- `esh-pve`
|
||||
- `esh-pve-nas`
|
||||
- `sfsrv-ana`
|
||||
|
||||
## API token (for hypervisor clients)
|
||||
|
||||
- `root@pam!fleet-vzdump` — stored in password manager; referenced
|
||||
by each hypervisor's `/etc/pve/storage.cfg` `pbs-ana` stanza.
|
||||
|
||||
## Sync to DR
|
||||
|
||||
One-way pull: **pbs-nh3** pulls from here at 06:00 daily. pbs-ana
|
||||
does not push; all coordination is on the DR side.
|
||||
|
||||
## Dependencies / blast radius
|
||||
|
||||
- **Hard dependency on ana-nas** (CT 109). Datastore NFS goes dead
|
||||
if ana-nas crashes. Fleet-wide vzdump + verify + sync all fail
|
||||
until NFS recovers. See `memory/incident_ana_nas_spof.md`.
|
||||
- **Self-backup excluded** from scheduled vzdump — backing up a
|
||||
PBS VM to itself is a circular dependency. Config-only backup
|
||||
could be captured via file-level restic if needed; not currently
|
||||
set up (low priority — the datastore is on NFS, OS is
|
||||
reinstallable in ~30 min).
|
||||
|
||||
## Refresh
|
||||
|
||||
```
|
||||
scripts/refresh-server-info.sh pbs-ana
|
||||
```
|
||||
|
||||
Snapshot at `system-details.txt`. Full deployment history:
|
||||
`docs/runbooks/pbs-deployment.md`.
|
||||
@@ -0,0 +1 @@
|
||||
lkraven@10.250.50.90
|
||||
@@ -0,0 +1,84 @@
|
||||
|
||||
===== HOST =====
|
||||
|
||||
Hostname: pbs-ana.phasefinal.com
|
||||
Date: 2026-04-23T21:55:34-07:00
|
||||
Uptime: up 7 hours, 26 minutes
|
||||
OS: Debian GNU/Linux 12 (bookworm)
|
||||
Kernel: 6.1.0-44-amd64
|
||||
Arch: x86_64
|
||||
|
||||
===== HARDWARE =====
|
||||
|
||||
CPU cores: 4
|
||||
CPU model: QEMU Virtual CPU version 2.5+
|
||||
MemTotal: 7.8 GB
|
||||
MemAvailable: 7.4 GB
|
||||
|
||||
===== GPUS =====
|
||||
|
||||
nvidia-smi not present (no NVIDIA GPUs or driver not installed)
|
||||
|
||||
===== FILESYSTEMS (df) =====
|
||||
|
||||
Filesystem Size Used Avail Use% Mounted on
|
||||
/dev/sda1 31G 2.2G 27G 8% /
|
||||
10.250.50.50:/mnt/backup/pbs-ana 20T 336G 19T 2% /mnt/pbs-datastore
|
||||
|
||||
===== PERSISTENT MOUNTS (/etc/fstab, non-comment) =====
|
||||
|
||||
UUID=6a2b24f3-4134-40ed-9b64-f708dcb77cef / ext4 errors=remount-ro 0 1
|
||||
UUID=bfe22def-87a6-4ca8-8f06-19bbf148559c none swap sw 0 0
|
||||
/dev/sr0 /media/cdrom0 udf,iso9660 user,noauto 0 0
|
||||
10.250.50.50:/mnt/backup/pbs-ana /mnt/pbs-datastore nfs defaults,_netdev,bg,hard,timeo=600,retrans=2,vers=3 0 0
|
||||
|
||||
===== TARGETED DATA PATHS =====
|
||||
|
||||
/opt (total: 4.0K)
|
||||
total 8
|
||||
drwxr-xr-x 2 root root 4096 2026-04-21 21:22 .
|
||||
drwxr-xr-x 18 root root 4096 2026-04-21 21:25 ..
|
||||
|
||||
/srv (total: 4.0K)
|
||||
total 8
|
||||
drwxr-xr-x 2 root root 4096 2026-04-21 21:22 .
|
||||
drwxr-xr-x 18 root root 4096 2026-04-21 21:25 ..
|
||||
|
||||
|
||||
===== DOCKER =====
|
||||
|
||||
docker not installed
|
||||
|
||||
===== COMPOSE FILES (/opt/docker/compose/) =====
|
||||
|
||||
/opt/docker/compose not present
|
||||
|
||||
===== CONFIG LAYOUT (/opt/docker/conf/ — top 200 entries) =====
|
||||
|
||||
/opt/docker/conf not present
|
||||
|
||||
===== LISTENING PORTS =====
|
||||
|
||||
0.0.0.0:111
|
||||
0.0.0.0:22
|
||||
0.0.0.0:36203
|
||||
0.0.0.0:36419
|
||||
[::]:111
|
||||
[::1]:25
|
||||
127.0.0.1:25
|
||||
127.0.0.1:82
|
||||
[::]:22
|
||||
[::]:35545
|
||||
[::]:56445
|
||||
*:8007
|
||||
|
||||
===== MODEL / HUGGINGFACE CACHES =====
|
||||
|
||||
|
||||
===== DOCKER-ADJACENT SYSTEMD SERVICES =====
|
||||
|
||||
(none matching)
|
||||
|
||||
===== DONE =====
|
||||
|
||||
Paste the above back into the chat, or pass a path as argv[1] to save.
|
||||
@@ -0,0 +1,99 @@
|
||||
# pbs-nh3
|
||||
|
||||
Proxmox Backup Server — **DR mirror**. VM on nh3-pve with an
|
||||
NFS-mounted datastore on the NH3 Synology. Pulls nightly from pbs-ana
|
||||
(the primary) at 06:00.
|
||||
|
||||
## Network
|
||||
|
||||
- **LAN IP:** 10.100.50.90
|
||||
- **FQDN:** `pbs-nh3.phasefinal.com`
|
||||
- **PBS web UI:** <https://10.100.50.90:8007/>
|
||||
- **SSH:** `ssh pbs-nh3` (config alias → `lkraven@10.100.50.90`).
|
||||
Root ops via sudo from lkraven.
|
||||
- **Homepage card:** *PBS-NH3* in `Infra - NH3` group.
|
||||
|
||||
## Hardware (VM)
|
||||
|
||||
- **Hypervisor:** nh3-pve
|
||||
- **vCPU:** 4 (QEMU)
|
||||
- **RAM:** 7.8 GB
|
||||
- **Root disk:** `/dev/sda1` 31 GB ext4 — OS only (2 GB used)
|
||||
- **Datastore:** **NFS from nh3-nas** at `/mnt/pbs-datastore` →
|
||||
`10.100.50.50:/volume1/pbs`. Volume size 42 TB; 27 TB used (64%).
|
||||
|
||||
## NFS details (the Synology ACL gotcha)
|
||||
|
||||
Synology's `/volume1/pbs` share was initially created with "Advanced
|
||||
Permissions" which imposes a `syno_acl` that denies writes to
|
||||
non-admin UIDs (even the `backup` uid PBS runs as). Fix was to flatten
|
||||
the share to pure POSIX mode 777 so the mount becomes "Linux mode"
|
||||
(`synoacltool -get` confirms). `no_root_squash + no_all_squash` also
|
||||
set on the NFS export.
|
||||
|
||||
Mount uses NFSv3 (same pattern as pbs-ana, for consistency more than
|
||||
necessity — Synology's Btrfs doesn't have the ZFS-case-insensitivity
|
||||
issue):
|
||||
|
||||
```
|
||||
10.100.50.50:/volume1/pbs /mnt/pbs-datastore nfs \
|
||||
defaults,_netdev,bg,hard,timeo=600,retrans=2,vers=3 0 0
|
||||
```
|
||||
|
||||
Full history: `docs/runbooks/pbs-deployment.md` Phase 5.3.
|
||||
|
||||
## What runs here
|
||||
|
||||
- `proxmox-backup-proxy` — HTTPS API + web UI (port 8007)
|
||||
- `proxmox-backup` — backend service
|
||||
- Scheduled verify job: weekly Sunday 12:00 (after nightly sync
|
||||
completes), `ignore-verified=true`, `outdated-after=30 days`
|
||||
- Scheduled sync from pbs-ana: daily at 06:00 UTC
|
||||
|
||||
## Remote config
|
||||
|
||||
- Remote name: `pbs-ana`
|
||||
- Remote host: `10.250.50.90:8007`
|
||||
- Auth: `root@pam!fleet-vzdump` token (same token as clients use on
|
||||
the primary — shared here for simplicity; could be split to a
|
||||
sync-only read-only token)
|
||||
- Cert fingerprint: in password manager under "pbs-ana fingerprint"
|
||||
|
||||
## Namespaces
|
||||
|
||||
Namespaces are auto-replicated from pbs-ana via the sync job — you do
|
||||
NOT need to pre-create them on the mirror. Expected namespaces after
|
||||
first sync: same 5 as pbs-ana (`pfi-pve`, `nh3-pve`, `esh-pve`,
|
||||
`esh-pve-nas`, `sfsrv-ana`).
|
||||
|
||||
## Dependencies / blast radius
|
||||
|
||||
- **Hard dependency on nh3-nas** (Synology). Datastore goes dead if
|
||||
the Synology crashes; sync fails, verify fails, restore ops fail.
|
||||
But: ANA primary keeps operating fine, so the fleet's backup SLA
|
||||
is preserved — only DR redundancy is temporarily lost.
|
||||
- **Sync source**: pbs-ana going offline means no new snapshots pull
|
||||
here, but existing ones remain restorable. Sync resumes
|
||||
automatically once pbs-ana recovers.
|
||||
|
||||
## DR failover (hypothetical)
|
||||
|
||||
If pbs-ana is permanently lost and we needed to restore fleet VMs
|
||||
from pbs-nh3:
|
||||
|
||||
1. Re-point each PVE host's `pbs-ana` storage entry at pbs-nh3
|
||||
(`pve-datacenter.cfg` → change `server` to 10.100.50.90)
|
||||
2. Fingerprint + token still work (same token)
|
||||
3. Cross-WAN restore is slower than local but functional
|
||||
|
||||
Documented as future work — not tested. Run a trial restore of a
|
||||
small CT periodically to validate.
|
||||
|
||||
## Refresh
|
||||
|
||||
```
|
||||
scripts/refresh-server-info.sh pbs-nh3
|
||||
```
|
||||
|
||||
Snapshot at `system-details.txt`. Full deployment history:
|
||||
`docs/runbooks/pbs-deployment.md`.
|
||||
@@ -0,0 +1 @@
|
||||
lkraven@10.100.50.90
|
||||
@@ -0,0 +1,84 @@
|
||||
|
||||
===== HOST =====
|
||||
|
||||
Hostname: pbs-nh3.phasefinal.com
|
||||
Date: 2026-04-23T21:56:13-07:00
|
||||
Uptime: up 22 hours, 41 minutes
|
||||
OS: Debian GNU/Linux 12 (bookworm)
|
||||
Kernel: 6.1.0-44-amd64
|
||||
Arch: x86_64
|
||||
|
||||
===== HARDWARE =====
|
||||
|
||||
CPU cores: 4
|
||||
CPU model: QEMU Virtual CPU version 2.5+
|
||||
MemTotal: 7.8 GB
|
||||
MemAvailable: 7.3 GB
|
||||
|
||||
===== GPUS =====
|
||||
|
||||
nvidia-smi not present (no NVIDIA GPUs or driver not installed)
|
||||
|
||||
===== FILESYSTEMS (df) =====
|
||||
|
||||
Filesystem Size Used Avail Use% Mounted on
|
||||
/dev/sda1 31G 2.0G 27G 7% /
|
||||
10.100.50.50:/volume1/pbs 42T 27T 16T 64% /mnt/pbs-datastore
|
||||
|
||||
===== PERSISTENT MOUNTS (/etc/fstab, non-comment) =====
|
||||
|
||||
UUID=34a61420-a414-4789-8a3d-065a8905eb11 / ext4 errors=remount-ro 0 1
|
||||
UUID=518afc03-e17d-4e1a-a5d4-84b96bee1c60 none swap sw 0 0
|
||||
/dev/sr0 /media/cdrom0 udf,iso9660 user,noauto 0 0
|
||||
10.100.50.50:/volume1/pbs /mnt/pbs-datastore nfs defaults,_netdev,bg,hard,timeo=600,retrans=2,vers=3 0 0
|
||||
|
||||
===== TARGETED DATA PATHS =====
|
||||
|
||||
/opt (total: 4.0K)
|
||||
total 8
|
||||
drwxr-xr-x 2 root root 4096 2026-04-22 21:04 .
|
||||
drwxr-xr-x 18 root root 4096 2026-04-22 21:06 ..
|
||||
|
||||
/srv (total: 4.0K)
|
||||
total 8
|
||||
drwxr-xr-x 2 root root 4096 2026-04-22 21:04 .
|
||||
drwxr-xr-x 18 root root 4096 2026-04-22 21:06 ..
|
||||
|
||||
|
||||
===== DOCKER =====
|
||||
|
||||
docker not installed
|
||||
|
||||
===== COMPOSE FILES (/opt/docker/compose/) =====
|
||||
|
||||
/opt/docker/compose not present
|
||||
|
||||
===== CONFIG LAYOUT (/opt/docker/conf/ — top 200 entries) =====
|
||||
|
||||
/opt/docker/conf not present
|
||||
|
||||
===== LISTENING PORTS =====
|
||||
|
||||
0.0.0.0:111
|
||||
0.0.0.0:22
|
||||
0.0.0.0:41669
|
||||
0.0.0.0:46653
|
||||
[::]:111
|
||||
[::1]:25
|
||||
127.0.0.1:25
|
||||
127.0.0.1:82
|
||||
[::]:22
|
||||
[::]:37707
|
||||
[::]:54621
|
||||
*:8007
|
||||
|
||||
===== MODEL / HUGGINGFACE CACHES =====
|
||||
|
||||
|
||||
===== DOCKER-ADJACENT SYSTEMD SERVICES =====
|
||||
|
||||
(none matching)
|
||||
|
||||
===== DONE =====
|
||||
|
||||
Paste the above back into the chat, or pass a path as argv[1] to save.
|
||||
@@ -0,0 +1,104 @@
|
||||
|
||||
===== HOST =====
|
||||
|
||||
Hostname: pfi-ana-webhost.phasefinal.com
|
||||
Date: 2026-04-23T01:00:53-07:00
|
||||
Uptime: up 34 weeks, 4 days, 11 hours, 35 minutes
|
||||
OS: Debian GNU/Linux 11 (bullseye)
|
||||
Kernel: 5.10.0-34-amd64
|
||||
Arch: x86_64
|
||||
|
||||
===== HARDWARE =====
|
||||
|
||||
CPU cores: 16
|
||||
CPU model: Common KVM processor
|
||||
MemTotal: 0.8 GB
|
||||
MemAvailable: 0.3 GB
|
||||
|
||||
===== GPUS =====
|
||||
|
||||
nvidia-smi not present (no NVIDIA GPUs or driver not installed)
|
||||
|
||||
===== FILESYSTEMS (df) =====
|
||||
|
||||
Filesystem Size Used Avail Use% Mounted on
|
||||
/dev/sda1 245G 8.6G 224G 4% /
|
||||
|
||||
===== PERSISTENT MOUNTS (/etc/fstab, non-comment) =====
|
||||
|
||||
UUID=110a7e2b-aaf8-4a55-a064-408b665ee322 / ext4 rw,grpquota,errors=remount-ro,relatime,quota,usrquota 0 1
|
||||
UUID=f788f710-a3ec-457e-8053-2eeca8c3f029 none swap sw 0 0
|
||||
/dev/sr0 /media/cdrom0 udf,iso9660 user,noauto 0 0
|
||||
/home/dragonfirepro /home/chroot/169559808468512/home/dragonfirepro bind defaults,bind 0
|
||||
|
||||
===== TARGETED DATA PATHS =====
|
||||
|
||||
/opt (total: 4.0K)
|
||||
total 8
|
||||
drwxr-xr-x 2 root root 4096 2023-09-24 15:42 .
|
||||
drwxr-xr-x 18 root root 4096 2025-04-12 13:58 ..
|
||||
|
||||
/srv (total: 12K)
|
||||
total 12
|
||||
drwxr-xr-x 3 root root 4096 2023-09-24 16:15 .
|
||||
drwxr-xr-x 18 root root 4096 2025-04-12 13:58 ..
|
||||
drwxr-xr-x 2 ftp nogroup 4096 2023-09-24 16:15 ftp
|
||||
|
||||
|
||||
===== DOCKER =====
|
||||
|
||||
docker not installed
|
||||
|
||||
===== COMPOSE FILES (/opt/docker/compose/) =====
|
||||
|
||||
/opt/docker/compose not present
|
||||
|
||||
===== CONFIG LAYOUT (/opt/docker/conf/ — top 200 entries) =====
|
||||
|
||||
/opt/docker/conf not present
|
||||
|
||||
===== LISTENING PORTS =====
|
||||
|
||||
0.0.0.0:10000
|
||||
0.0.0.0:110
|
||||
0.0.0.0:143
|
||||
0.0.0.0:20000
|
||||
0.0.0.0:22
|
||||
0.0.0.0:993
|
||||
0.0.0.0:995
|
||||
[::]:10000
|
||||
10.250.50.52:53
|
||||
[::]:110
|
||||
[::1]:10023
|
||||
[::1]:25
|
||||
127.0.0.1:10023
|
||||
127.0.0.1:25
|
||||
127.0.0.1:3306
|
||||
127.0.0.1:465
|
||||
127.0.0.1:53
|
||||
127.0.0.1:587
|
||||
127.0.0.1:953
|
||||
[::]:143
|
||||
[::1]:465
|
||||
[::1]:53
|
||||
[::1]:587
|
||||
[::1]:953
|
||||
*:21
|
||||
[::]:22
|
||||
*:2222
|
||||
*:443
|
||||
*:80
|
||||
[::]:993
|
||||
[::]:995
|
||||
[fe80::e4f9:3aff:fec9:612a]%ens18:53
|
||||
|
||||
===== MODEL / HUGGINGFACE CACHES =====
|
||||
|
||||
|
||||
===== DOCKER-ADJACENT SYSTEMD SERVICES =====
|
||||
|
||||
(none matching)
|
||||
|
||||
===== DONE =====
|
||||
|
||||
Paste the above back into the chat, or pass a path as argv[1] to save.
|
||||
@@ -0,0 +1,80 @@
|
||||
|
||||
===== HOST =====
|
||||
|
||||
Hostname: pfi-postgres.phasefinal.com
|
||||
Date: 2026-04-23T01:01:01-07:00
|
||||
Uptime: up 34 weeks, 4 days, 5 hours, 12 minutes
|
||||
OS: Debian GNU/Linux 12 (bookworm)
|
||||
Kernel: 6.1.0-38-amd64
|
||||
Arch: x86_64
|
||||
|
||||
===== HARDWARE =====
|
||||
|
||||
CPU cores: 16
|
||||
CPU model: Common KVM processor
|
||||
MemTotal: 7.8 GB
|
||||
MemAvailable: 7.2 GB
|
||||
|
||||
===== GPUS =====
|
||||
|
||||
nvidia-smi not present (no NVIDIA GPUs or driver not installed)
|
||||
|
||||
===== FILESYSTEMS (df) =====
|
||||
|
||||
Filesystem Size Used Avail Use% Mounted on
|
||||
/dev/sda1 78G 4.2G 70G 6% /
|
||||
10.250.50.50:/mnt/db 20T 281M 20T 1% /mnt/db
|
||||
|
||||
===== PERSISTENT MOUNTS (/etc/fstab, non-comment) =====
|
||||
|
||||
UUID=e4f13bda-0320-43e7-be36-8df370f5c099 / ext4 errors=remount-ro 0 1
|
||||
UUID=f9efdef0-743c-46b9-8131-ae0a3e07e40a none swap sw 0 0
|
||||
/dev/sr0 /media/cdrom0 udf,iso9660 user,noauto 0 0
|
||||
10.250.50.50:/mnt/db /mnt/db nfs defaults 0 0
|
||||
|
||||
===== TARGETED DATA PATHS =====
|
||||
|
||||
/opt (total: 4.0K)
|
||||
total 8
|
||||
drwxr-xr-x 2 root root 4096 2023-04-30 10:11 .
|
||||
drwxr-xr-x 18 root root 4096 2025-08-23 19:46 ..
|
||||
|
||||
/srv (total: 4.0K)
|
||||
total 8
|
||||
drwxr-xr-x 2 root root 4096 2023-04-30 10:11 .
|
||||
drwxr-xr-x 18 root root 4096 2025-08-23 19:46 ..
|
||||
|
||||
|
||||
===== DOCKER =====
|
||||
|
||||
docker not installed
|
||||
|
||||
===== COMPOSE FILES (/opt/docker/compose/) =====
|
||||
|
||||
/opt/docker/compose not present
|
||||
|
||||
===== CONFIG LAYOUT (/opt/docker/conf/ — top 200 entries) =====
|
||||
|
||||
/opt/docker/conf not present
|
||||
|
||||
===== LISTENING PORTS =====
|
||||
|
||||
0.0.0.0:111
|
||||
0.0.0.0:22
|
||||
0.0.0.0:5432
|
||||
[::]:111
|
||||
127.0.0.1:5433
|
||||
[::1]:5433
|
||||
[::]:22
|
||||
[::]:5432
|
||||
|
||||
===== MODEL / HUGGINGFACE CACHES =====
|
||||
|
||||
|
||||
===== DOCKER-ADJACENT SYSTEMD SERVICES =====
|
||||
|
||||
(none matching)
|
||||
|
||||
===== DONE =====
|
||||
|
||||
Paste the above back into the chat, or pass a path as argv[1] to save.
|
||||
@@ -0,0 +1,100 @@
|
||||
|
||||
===== HOST =====
|
||||
|
||||
Hostname: pfi-pteradactyl.phasefinal.com
|
||||
Date: 2026-04-23T01:00:55-07:00
|
||||
Uptime: up 34 weeks, 4 days, 11 hours, 33 minutes
|
||||
OS: Debian GNU/Linux 12 (bookworm)
|
||||
Kernel: 6.1.0-34-amd64
|
||||
Arch: x86_64
|
||||
|
||||
===== HARDWARE =====
|
||||
|
||||
CPU cores: 8
|
||||
CPU model: Intel(R) Xeon(R) Silver 4310 CPU @ 2.10GHz
|
||||
MemTotal: 7.8 GB
|
||||
MemAvailable: 0.3 GB
|
||||
|
||||
===== GPUS =====
|
||||
|
||||
nvidia-smi not present (no NVIDIA GPUs or driver not installed)
|
||||
|
||||
===== FILESYSTEMS (df) =====
|
||||
|
||||
Filesystem Size Used Avail Use% Mounted on
|
||||
/dev/sda1 250G 67G 172G 28% /
|
||||
|
||||
===== PERSISTENT MOUNTS (/etc/fstab, non-comment) =====
|
||||
|
||||
UUID=028d611a-2cbc-4523-b1f3-2aa04d74302e / ext4 errors=remount-ro 0 1
|
||||
UUID=578e5418-a021-407d-8162-4b725807ee13 none swap sw 0 0
|
||||
/dev/sr0 /media/cdrom0 udf,iso9660 user,noauto 0 0
|
||||
|
||||
===== TARGETED DATA PATHS =====
|
||||
|
||||
/opt (total: 4.0K)
|
||||
total 8
|
||||
drwxr-xr-x 2 root root 4096 2023-05-14 13:22 .
|
||||
drwxr-xr-x 18 root root 4096 2025-04-28 17:36 ..
|
||||
|
||||
/var/lib/docker (total: 4.0K)
|
||||
|
||||
/srv (total: 4.0K)
|
||||
total 8
|
||||
drwxr-xr-x 2 root root 4096 2023-05-14 13:22 .
|
||||
drwxr-xr-x 18 root root 4096 2025-04-28 17:36 ..
|
||||
|
||||
|
||||
===== DOCKER =====
|
||||
|
||||
Server:
|
||||
docker daemon not reachable by current user
|
||||
|
||||
----- docker info -----
|
||||
Containers:
|
||||
|
||||
----- running containers -----
|
||||
|
||||
----- all containers -----
|
||||
|
||||
----- networks -----
|
||||
|
||||
----- networks (external, non-default — worth knowing for compose external: true) -----
|
||||
|
||||
----- named volumes -----
|
||||
|
||||
----- compose projects currently running -----
|
||||
(none)
|
||||
|
||||
===== COMPOSE FILES (/opt/docker/compose/) =====
|
||||
|
||||
/opt/docker/compose not present
|
||||
|
||||
===== CONFIG LAYOUT (/opt/docker/conf/ — top 200 entries) =====
|
||||
|
||||
/opt/docker/conf not present
|
||||
|
||||
===== LISTENING PORTS =====
|
||||
|
||||
0.0.0.0:22
|
||||
0.0.0.0:80
|
||||
10.250.50.55:7777
|
||||
127.0.0.1:3306
|
||||
127.0.0.1:34135
|
||||
127.0.0.1:6379
|
||||
[::1]:6379
|
||||
*:2022
|
||||
[::]:22
|
||||
*:8080
|
||||
|
||||
===== MODEL / HUGGINGFACE CACHES =====
|
||||
|
||||
|
||||
===== DOCKER-ADJACENT SYSTEMD SERVICES =====
|
||||
|
||||
containerd.service running
|
||||
docker.service running
|
||||
|
||||
===== DONE =====
|
||||
|
||||
Paste the above back into the chat, or pass a path as argv[1] to save.
|
||||
@@ -4,13 +4,16 @@ Proxmox VE hypervisor for the Anaheim colo (`pve.phasefinal.com`). Hosts every V
|
||||
|
||||
## Network
|
||||
|
||||
- **LAN IP:** 10.250.250.31
|
||||
- **LAN IP:** 10.250.250.31 (in-band, OS-side)
|
||||
- **iDRAC (OOB):** 10.250.250.30 — Dell iDRAC web UI at
|
||||
<https://10.250.250.30/> (homepage card: *PFI-r750xs-iDRAC*)
|
||||
- **FQDN:** `pve.phasefinal.com`
|
||||
- **Web UI:** `https://10.250.250.31:8006`
|
||||
- **SSH:** `root@pfi-pve` (key auth; `ssh-target` file uses the hostname)
|
||||
|
||||
## Hardware
|
||||
|
||||
- **Chassis:** Dell PowerEdge R750xs (2U rackmount)
|
||||
- **CPU:** Intel Xeon Silver 4310 @ 2.10 GHz
|
||||
- **RAM:** 188 GB
|
||||
- **Kernel:** `6.8.12-8-pve` (Proxmox 8.x)
|
||||
@@ -18,9 +21,24 @@ Proxmox VE hypervisor for the Anaheim colo (`pve.phasefinal.com`). Hosts every V
|
||||
|
||||
## What it runs
|
||||
|
||||
The VMs in [`docs/pfi/proxmox-vms.md`](../../docs/pfi/proxmox-vms.md) live here — VM 100 (PFI-ANA-TRUENAS, now Debian NFS/SMB), VM 101 (DC), VM 102 (PFI-ANA-Docker = `ana-docker`), 103–111 (Mongo, Postgres, Tailscale, Pteradactyl, Webhost, tacticalrmm, etc.).
|
||||
The VMs/CTs in [`docs/pfi/proxmox-vms.md`](../../docs/pfi/proxmox-vms.md)
|
||||
live here. Known as of 2026-04-23 (from vzdump logs + `servers/`
|
||||
dirs; not exhaustive):
|
||||
|
||||
`ana-ml2` (`10.250.50.54`) is a separate bare-metal box, not on this hypervisor.
|
||||
- **VM 102** — `ana-docker` (PFI-ANA-Docker)
|
||||
- **VM 105** — `pfi-postgres` (PFI-Postgres; PGDATA on NFS from ana-nas)
|
||||
- **VM 107** — `pfi-pteradactyl` (game panel)
|
||||
- **VM 110** — `pfi-ana-webhost`
|
||||
- **VM 111** — `pfi-tacticalrmm`
|
||||
- **VM (TBD)** — `pbs-ana` (Proxmox Backup Server; 10.250.50.90)
|
||||
- **CT 109** — `ana-nas` (Debian LXC serving NFS/SMB; bind-mounts
|
||||
from ospool; data-serving SPOF — see `memory/incident_ana_nas_spof.md`)
|
||||
- **CT 112** — `ana-filebot` (file-task automation)
|
||||
- **CT 113** — `ana-wg` (WireGuard VPN)
|
||||
|
||||
`ana-ml2` (`10.250.50.54`) is a separate bare-metal Supermicro box,
|
||||
not on this hypervisor. `sfsrv-ana` (`10.250.250.115`) runs on a
|
||||
separate Dell R630, also not on this hypervisor.
|
||||
|
||||
## Refresh state
|
||||
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
|
||||
===== HOST =====
|
||||
|
||||
Hostname: pfi-tacticalrmm.phasefinal.com
|
||||
Date: 2026-04-23T01:01:00-07:00
|
||||
Uptime: up 2 days, 6 hours, 0 minutes
|
||||
OS: Debian GNU/Linux 12 (bookworm)
|
||||
Kernel: 6.1.0-32-amd64
|
||||
Arch: x86_64
|
||||
|
||||
===== HARDWARE =====
|
||||
|
||||
CPU cores: 16
|
||||
CPU model: Common KVM processor
|
||||
MemTotal: 7.8 GB
|
||||
MemAvailable: 6.0 GB
|
||||
|
||||
===== GPUS =====
|
||||
|
||||
nvidia-smi not present (no NVIDIA GPUs or driver not installed)
|
||||
|
||||
===== FILESYSTEMS (df) =====
|
||||
|
||||
Filesystem Size Used Avail Use% Mounted on
|
||||
/dev/sda1 250G 7.1G 231G 3% /
|
||||
|
||||
===== PERSISTENT MOUNTS (/etc/fstab, non-comment) =====
|
||||
|
||||
UUID=ce4c34ea-d1fa-46ea-90c9-5538c1a077c4 / ext4 errors=remount-ro 0 1
|
||||
UUID=125637a3-f2eb-492f-85ad-96c4bbda1387 none swap sw 0 0
|
||||
/dev/sr0 /media/cdrom0 udf,iso9660 user,noauto 0 0
|
||||
|
||||
===== TARGETED DATA PATHS =====
|
||||
|
||||
/opt (total: 9.8M)
|
||||
total 16
|
||||
drwxr-xr-x 4 root root 4096 2023-11-02 12:34 .
|
||||
drwxr-xr-x 20 root root 4096 2025-04-07 14:39 ..
|
||||
drwxr-xr-x 3 tactical tactical 4096 2023-11-02 12:34 tactical
|
||||
drwxr-xr-x 8 tactical tactical 4096 2026-04-20 20:37 trmm-community-scripts
|
||||
|
||||
/srv (total: 4.0K)
|
||||
total 8
|
||||
drwxr-xr-x 2 root root 4096 2023-11-02 11:18 .
|
||||
drwxr-xr-x 20 root root 4096 2025-04-07 14:39 ..
|
||||
|
||||
|
||||
===== DOCKER =====
|
||||
|
||||
docker not installed
|
||||
|
||||
===== COMPOSE FILES (/opt/docker/compose/) =====
|
||||
|
||||
/opt/docker/compose not present
|
||||
|
||||
===== CONFIG LAYOUT (/opt/docker/conf/ — top 200 entries) =====
|
||||
|
||||
/opt/docker/conf not present
|
||||
|
||||
===== LISTENING PORTS =====
|
||||
|
||||
0.0.0.0:22
|
||||
0.0.0.0:443
|
||||
0.0.0.0:80
|
||||
*:1024
|
||||
127.0.0.1:4222
|
||||
127.0.0.1:5432
|
||||
127.0.0.1:6379
|
||||
127.0.0.1:9235
|
||||
[::1]:5432
|
||||
[::1]:6379
|
||||
[::]:22
|
||||
[::]:443
|
||||
*:4430
|
||||
*:4433
|
||||
[::]:80
|
||||
|
||||
===== MODEL / HUGGINGFACE CACHES =====
|
||||
|
||||
|
||||
===== DOCKER-ADJACENT SYSTEMD SERVICES =====
|
||||
|
||||
(none matching)
|
||||
|
||||
===== DONE =====
|
||||
|
||||
Paste the above back into the chat, or pass a path as argv[1] to save.
|
||||
@@ -0,0 +1,249 @@
|
||||
|
||||
===== HOST =====
|
||||
|
||||
Hostname: sf-ana-container.surefire.com
|
||||
Date: 2026-04-21T16:57:55-07:00
|
||||
Uptime: up 3 weeks, 6 days, 1 hour, 17 minutes
|
||||
OS: Debian GNU/Linux 12 (bookworm)
|
||||
Kernel: 6.1.0-21-amd64
|
||||
Arch: x86_64
|
||||
|
||||
===== HARDWARE =====
|
||||
|
||||
CPU cores: 16
|
||||
CPU model: QEMU Virtual CPU version 2.5+
|
||||
MemTotal: 15.6 GB
|
||||
MemAvailable: 14.4 GB
|
||||
|
||||
===== GPUS =====
|
||||
|
||||
nvidia-smi not present (no NVIDIA GPUs or driver not installed)
|
||||
|
||||
===== FILESYSTEMS (df) =====
|
||||
|
||||
Filesystem Size Used Avail Use% Mounted on
|
||||
/dev/sda1 250G 7.1G 231G 3% /
|
||||
|
||||
===== PERSISTENT MOUNTS (/etc/fstab, non-comment) =====
|
||||
|
||||
UUID=fcc00570-1585-41d4-8c64-8e5d59af40e0 / ext4 errors=remount-ro 0 1
|
||||
UUID=35ad80f9-5952-489a-9178-575be588420c none swap sw 0 0
|
||||
/dev/sr0 /media/cdrom0 udf,iso9660 user,noauto 0 0
|
||||
|
||||
===== TARGETED DATA PATHS =====
|
||||
|
||||
/opt (total: 8.3M)
|
||||
total 12
|
||||
drwxr-xr-x 3 root root 4096 2024-06-07 10:42 .
|
||||
drwxr-xr-x 18 root root 4096 2024-06-07 10:35 ..
|
||||
drwxrwxrwx 5 root root 4096 2024-06-07 10:46 docker
|
||||
|
||||
/opt/docker (total: 8.3M)
|
||||
total 20
|
||||
drwxrwxrwx 5 root root 4096 2024-06-07 10:46 .
|
||||
drwxr-xr-x 3 root root 4096 2024-06-07 10:42 ..
|
||||
drwxr-xr-x 6 vhoang vhoang 4096 2024-06-07 11:09 compose
|
||||
drwxr-xr-x 3 vhoang vhoang 4096 2024-06-07 10:46 conf
|
||||
drwxr-xr-x 2 vhoang vhoang 4096 2024-06-07 10:42 data
|
||||
|
||||
/opt/docker/compose (total: 8.3M)
|
||||
total 24
|
||||
drwxr-xr-x 6 vhoang vhoang 4096 2024-06-07 11:09 .
|
||||
drwxrwxrwx 5 root root 4096 2024-06-07 10:46 ..
|
||||
drwxr-xr-x 2 vhoang vhoang 4096 2024-06-07 10:43 dockge
|
||||
drwxr-xr-x 4 root root 4096 2024-06-07 10:58 nginx-proxy
|
||||
drwxr-xr-x 2 root root 4096 2024-06-07 10:46 traefik
|
||||
drwxr-xr-x 3 root root 4096 2024-06-07 11:09 vaultwarden
|
||||
|
||||
/opt/docker/conf (total: 8.0K)
|
||||
total 12
|
||||
drwxr-xr-x 3 vhoang vhoang 4096 2024-06-07 10:46 .
|
||||
drwxrwxrwx 5 root root 4096 2024-06-07 10:46 ..
|
||||
drwxr-xr-x 2 root root 4096 2024-06-07 10:46 traefik-esh
|
||||
|
||||
/var/lib/docker (total: 4.0K)
|
||||
|
||||
/srv (total: 4.0K)
|
||||
total 8
|
||||
drwxr-xr-x 2 root root 4096 2023-11-09 17:45 .
|
||||
drwxr-xr-x 18 root root 4096 2024-06-07 10:35 ..
|
||||
|
||||
|
||||
===== DOCKER =====
|
||||
|
||||
Server:
|
||||
docker daemon not reachable by current user
|
||||
|
||||
----- docker info -----
|
||||
Containers:
|
||||
|
||||
----- running containers -----
|
||||
|
||||
----- all containers -----
|
||||
|
||||
----- networks -----
|
||||
|
||||
----- networks (external, non-default — worth knowing for compose external: true) -----
|
||||
|
||||
----- named volumes -----
|
||||
|
||||
----- compose projects currently running -----
|
||||
(none)
|
||||
|
||||
===== COMPOSE FILES (/opt/docker/compose/) =====
|
||||
|
||||
|
||||
>>> /opt/docker/compose/dockge/compose.yaml
|
||||
services:
|
||||
dockge:
|
||||
image: louislam/dockge:latest
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
# Host Port : Container Port
|
||||
- 5001:5001
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock
|
||||
- dockge_data:/app/data
|
||||
- /opt/docker/compose:/opt/docker/compose
|
||||
environment:
|
||||
# Tell Dockge where is your stacks directory
|
||||
- DOCKGE_STACKS_DIR=/opt/docker/compose
|
||||
networks:
|
||||
- tnet
|
||||
|
||||
volumes:
|
||||
dockge_data: null
|
||||
networks:
|
||||
tnet:
|
||||
name: traefik-net
|
||||
external: true
|
||||
|
||||
>>> /opt/docker/compose/nginx-proxy/compose.yaml
|
||||
version: "3.8"
|
||||
services:
|
||||
app:
|
||||
image: jc21/nginx-proxy-manager:latest
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
# These ports are in format <host-port>:<container-port>
|
||||
- 80:80 # Public HTTP Port
|
||||
- 443:443 # Public HTTPS Port
|
||||
- 81:81 # Admin Web Port
|
||||
# Add any other Stream port you want to expose
|
||||
# - '21:21' # FTP
|
||||
|
||||
# Uncomment the next line if you uncomment anything in the section
|
||||
# environment:
|
||||
# Uncomment this if you want to change the location of
|
||||
# the SQLite DB file within the container
|
||||
# DB_SQLITE_FILE: "/data/database.sqlite"
|
||||
|
||||
# Uncomment this if IPv6 is not enabled on your host
|
||||
# DISABLE_IPV6: 'true'
|
||||
|
||||
volumes:
|
||||
- ./data:/data
|
||||
- ./letsencrypt:/etc/letsencrypt
|
||||
|
||||
networks:
|
||||
default:
|
||||
external: true
|
||||
name: nginx-proxy_default
|
||||
|
||||
>>> /opt/docker/compose/traefik/compose.yaml
|
||||
#version: "3.3"
|
||||
|
||||
services:
|
||||
traefik:
|
||||
image: traefik:latest
|
||||
container_name: traefik
|
||||
command:
|
||||
- --log.level=DEBUG
|
||||
- --configFile=/etc/traefik/traefik.yml
|
||||
- --api.insecure=true
|
||||
- --providers.docker=true
|
||||
- --providers.docker.exposedbydefault=false
|
||||
- --entrypoints.web.address=:80
|
||||
environment:
|
||||
- AWS_ACCESS_KEY_ID=${AWS_KEY}
|
||||
- AWS_SECRET_ACCESS_KEY=${AWS_SECRET}
|
||||
- AWS_REGION=${AWS_REGION}
|
||||
- AWS_HOSTED_ZONE_ID=${AWS_ZONEID}
|
||||
ports:
|
||||
- 80:80
|
||||
- 8380:8080
|
||||
- 443:443
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||
- /opt/docker/conf/traefik-esh:/etc/traefik
|
||||
networks:
|
||||
- tnet
|
||||
env_file:
|
||||
- .env
|
||||
networks:
|
||||
tnet:
|
||||
name: traefik-net
|
||||
external: true
|
||||
|
||||
>>> /opt/docker/compose/vaultwarden/compose.yaml
|
||||
version: "3"
|
||||
services:
|
||||
vaultwarden:
|
||||
image: vaultwarden/server:latest
|
||||
container_name: vaultwarden
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- 9445:80 #map any custom port to use (replace 8445 not 80)
|
||||
volumes:
|
||||
- ./bitwarden:/data:rw
|
||||
environment:
|
||||
- ADMIN_TOKEN=${ADMIN_TOKEN}
|
||||
- WEBSOCKET_ENABLED=true
|
||||
- SIGNUPS_ALLOWED=true
|
||||
- DOMAIN=${DOMAIN}
|
||||
- SMTP_HOST=surefire-com.mail.protection.outlook.com
|
||||
- SMTP_FROM=vaultwarden@surefire.com
|
||||
- SMTP_PORT=25
|
||||
# - SMTP_USERNAME=relay@surefire.com
|
||||
# - SMTP_PASSWORD=5Gb&237N@p%6
|
||||
networks:
|
||||
default:
|
||||
external: true
|
||||
name: nginx-proxy_default
|
||||
|
||||
===== CONFIG LAYOUT (/opt/docker/conf/ — top 200 entries) =====
|
||||
|
||||
/opt/docker/conf
|
||||
/opt/docker/conf/traefik-esh
|
||||
|
||||
===== LISTENING PORTS =====
|
||||
|
||||
0.0.0.0:22
|
||||
0.0.0.0:443
|
||||
0.0.0.0:5001
|
||||
0.0.0.0:80
|
||||
0.0.0.0:8000
|
||||
0.0.0.0:81
|
||||
0.0.0.0:9443
|
||||
0.0.0.0:9445
|
||||
127.0.0.1:46567
|
||||
[::]:22
|
||||
[::]:443
|
||||
[::]:5001
|
||||
[::]:80
|
||||
[::]:8000
|
||||
[::]:81
|
||||
[::]:9443
|
||||
[::]:9445
|
||||
|
||||
===== MODEL / HUGGINGFACE CACHES =====
|
||||
|
||||
|
||||
===== DOCKER-ADJACENT SYSTEMD SERVICES =====
|
||||
|
||||
containerd.service running
|
||||
docker.service running
|
||||
|
||||
===== DONE =====
|
||||
|
||||
Paste the above back into the chat, or pass a path as argv[1] to save.
|
||||
+30
-21
@@ -3,43 +3,52 @@
|
||||
SureFire-client physical server at the Anaheim colo — Dell PowerEdge
|
||||
R630. **PFI-managed** under the hosting agreement (hardware + OS ops).
|
||||
|
||||
**This directory documents the hardware / iDRAC side only.** The OS
|
||||
running on this box is tracked separately as
|
||||
[`servers/sfsrv-ana/`](../sfsrv-ana/README.md) — Proxmox VE at
|
||||
`10.250.250.115`. Refer to that for OS-level snapshots, SSH access,
|
||||
and backup coverage.
|
||||
|
||||
## Client context
|
||||
|
||||
- **Client:** SureFire
|
||||
- **PFI role:** full-service managed host
|
||||
- **PFI role:** full-service managed host (iDRAC + Proxmox)
|
||||
|
||||
## Network
|
||||
|
||||
- **iDRAC BMC IP:** 10.250.250.110 (on the management subnet)
|
||||
- Web UI: https://10.250.250.110/
|
||||
- **OS-side LAN IP:** **unknown** — not in FortiGate DHCP (static
|
||||
config?). Fill this in once identified.
|
||||
- **SSH:** no `ssh-target` set yet because the OS IP isn't known. Add
|
||||
one (e.g. `root@<ip>` or `lkraven@<ip>`) once identified.
|
||||
- **iDRAC BMC:** `10.250.250.110` (dedicated OOB management NIC)
|
||||
- Web UI: <https://10.250.250.110/> (currently 404s; use the
|
||||
vSphere-style Redfish API or racadm from iDRAC shell for
|
||||
out-of-band ops)
|
||||
- Homepage monitors this via ICMP ping rather than HTTP
|
||||
- **OS-side (Proxmox VE):** see `servers/sfsrv-ana/` — `10.250.250.115`
|
||||
|
||||
## Infrastructure
|
||||
## Hardware
|
||||
|
||||
- **Type:** Physical Dell PowerEdge R630
|
||||
- **Type:** Physical Dell PowerEdge R630, 2U
|
||||
- **Site:** Anaheim (PFI colo)
|
||||
- Detailed hardware / CPU / RAM / disk data comes from `sfsrv-ana`'s
|
||||
`system-details.txt` (the OS can read all of this via `dmidecode`,
|
||||
`lscpu`, etc. — no need to duplicate here).
|
||||
|
||||
## Backup coverage
|
||||
## Why this dir exists separately from sfsrv-ana
|
||||
|
||||
- **VM-image:** N/A (bare-metal physical)
|
||||
- **File-level restic:** not yet configured
|
||||
- Pending OS identification + SSH access
|
||||
- Follow the `configs/restic/ana-ml2/` template when setting up
|
||||
(both are bare-metal hosts; similar source list + no DB hooks by
|
||||
default unless SF apps run databases)
|
||||
- The iDRAC is a distinct management surface with its own credentials,
|
||||
protocol stack, and firmware lifecycle. Power events, BIOS updates,
|
||||
and console access live here.
|
||||
- Inventory audits that track *hardware* (for hosting-agreement
|
||||
billing, depreciation, etc.) should see the physical box as its own
|
||||
entry independent of whatever OS happens to be running on it.
|
||||
|
||||
## Refresh state
|
||||
|
||||
Once the OS IP is known and an `ssh-target` is in place:
|
||||
```bash
|
||||
scripts/refresh-server-info.sh sf-r630
|
||||
```
|
||||
**No `ssh-target` and no `system-details.txt`** — iDRAC doesn't expose
|
||||
a useful sshd for our inspect scripts. Host metadata comes from
|
||||
`sfsrv-ana`'s snapshot instead.
|
||||
|
||||
## Discovered via
|
||||
|
||||
`scripts/discover-fortigate.sh 10.250.250.1` on 2026-04-21 — iDRAC's
|
||||
DHCP lease on the management interface (MAC `74:e6:e2:fe:2c:7c`,
|
||||
VCI `iDRAC`).
|
||||
VCI `iDRAC`). Matched to sfsrv-ana via iDRAC Service Tag / physical
|
||||
co-location on 2026-04-23.
|
||||
|
||||
@@ -0,0 +1,164 @@
|
||||
|
||||
===== HOST =====
|
||||
|
||||
Hostname: sfsrv-ana.surefire.com
|
||||
Date: 2026-04-21T16:57:44-07:00
|
||||
Uptime: up 3 weeks, 6 days, 1 hour, 17 minutes
|
||||
PVE: pve-manager/8.2.2/9355359cd7afbae4 (running kernel: 6.8.4-3-pve)
|
||||
Kernel: 6.8.4-3-pve
|
||||
Arch: x86_64
|
||||
|
||||
===== HARDWARE =====
|
||||
|
||||
CPU model: Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
|
||||
CPU cores: 24
|
||||
MemTotal: 110.0 GB
|
||||
MemAvail: 104.1 GB
|
||||
|
||||
===== CLUSTER =====
|
||||
|
||||
(single-node host (or pvecm returned no cluster info))
|
||||
|
||||
===== CLUSTER RESOURCES (pvesh) =====
|
||||
|
||||
┌─────────────────────────────┬─────────┬─────────────┬───────────────────┬───────┬───────────┬─────────┬───────┬────────┬────────────┬────────────┬──────────┬──────────────────┬───────────┬────────────┬──────┬───────────┬───────────┬──────────┬──────┐
|
||||
│ id │ type │ cgroup-mode │ content │ cpu │ disk │ hastate │ level │ maxcpu │ maxdisk │ maxmem │ mem │ name │ node │ plugintype │ pool │ status │ storage │ uptime │ vmid │
|
||||
╞═════════════════════════════╪═════════╪═════════════╪═══════════════════╪═══════╪═══════════╪═════════╪═══════╪════════╪════════════╪════════════╪══════════╪══════════════════╪═══════════╪════════════╪══════╪═══════════╪═══════════╪══════════╪══════╡
|
||||
│ node/sfsrv-ana │ node │ 2 │ │ 0.32% │ 6.71 GiB │ │ │ 24 │ 93.93 GiB │ 110.05 GiB │ 5.05 GiB │ │ sfsrv-ana │ │ │ online │ │ 3w 6d 1h │ │
|
||||
├─────────────────────────────┼─────────┼─────────────┼───────────────────┼───────┼───────────┼─────────┼───────┼────────┼────────────┼────────────┼──────────┼──────────────────┼───────────┼────────────┼──────┼───────────┼───────────┼──────────┼──────┤
|
||||
│ qemu/100 │ qemu │ │ │ 0.33% │ 0.00 B │ │ │ 16 │ 256.00 GiB │ 16.00 GiB │ 2.99 GiB │ sf-ana-container │ sfsrv-ana │ │ │ running │ │ 3w 6d 1h │ 100 │
|
||||
├─────────────────────────────┼─────────┼─────────────┼───────────────────┼───────┼───────────┼─────────┼───────┼────────┼────────────┼────────────┼──────────┼──────────────────┼───────────┼────────────┼──────┼───────────┼───────────┼──────────┼──────┤
|
||||
│ sdn/sfsrv-ana/localnetwork │ sdn │ │ │ │ │ │ │ │ │ │ │ │ sfsrv-ana │ │ │ ok │ │ │ │
|
||||
├─────────────────────────────┼─────────┼─────────────┼───────────────────┼───────┼───────────┼─────────┼───────┼────────┼────────────┼────────────┼──────────┼──────────────────┼───────────┼────────────┼──────┼───────────┼───────────┼──────────┼──────┤
|
||||
│ storage/sfsrv-ana/local │ storage │ │ backup,iso,vztmpl │ │ 6.71 GiB │ │ │ │ 93.93 GiB │ │ │ │ sfsrv-ana │ dir │ │ available │ local │ │ │
|
||||
├─────────────────────────────┼─────────┼─────────────┼───────────────────┼───────┼───────────┼─────────┼───────┼────────┼────────────┼────────────┼──────────┼──────────────────┼───────────┼────────────┼──────┼───────────┼───────────┼──────────┼──────┤
|
||||
│ storage/sfsrv-ana/local-lvm │ storage │ │ rootdir,images │ │ 13.97 GiB │ │ │ │ 793.79 GiB │ │ │ │ sfsrv-ana │ lvmthin │ │ available │ local-lvm │ │ │
|
||||
└─────────────────────────────┴─────────┴─────────────┴───────────────────┴───────┴───────────┴─────────┴───────┴────────┴────────────┴────────────┴──────────┴──────────────────┴───────────┴────────────┴──────┴───────────┴───────────┴──────────┴──────┘
|
||||
|
||||
===== VMs (qm list) =====
|
||||
|
||||
VMID NAME STATUS MEM(MB) BOOTDISK(GB) PID
|
||||
100 sf-ana-container running 16384 256.00 1361
|
||||
|
||||
===== VM CONFIGS (qm config per id) =====
|
||||
|
||||
|
||||
----- VMID 100 -----
|
||||
agent: 1
|
||||
balloon: 0
|
||||
boot: order=scsi0;ide2;net0
|
||||
cores: 4
|
||||
ide2: local:iso/debian-12.2.0-amd64-netinst.iso,media=cdrom,size=628M
|
||||
memory: 16384
|
||||
name: sf-ana-container
|
||||
net0: virtio=6A:09:9A:8A:08:A5,bridge=vmbr0,firewall=1,tag=150
|
||||
onboot: 1
|
||||
ostype: l26
|
||||
scsi0: local-lvm:vm-100-disk-0,iothread=1,size=256G
|
||||
|
||||
===== LXC CONTAINERS (pct list) =====
|
||||
|
||||
|
||||
===== LXC CONFIGS (pct config per id) =====
|
||||
|
||||
(no LXC containers found)
|
||||
|
||||
===== STORAGE (pvesm status) =====
|
||||
|
||||
Name Type Status Total Used Available %
|
||||
local dir active 98497780 7034220 86414012 7.14%
|
||||
local-lvm lvmthin active 832352256 14649399 817702856 1.76%
|
||||
|
||||
===== DATASTORE CONFIG (/etc/pve/storage.cfg) =====
|
||||
|
||||
dir: local
|
||||
path /var/lib/vz
|
||||
content iso,vztmpl,backup
|
||||
|
||||
lvmthin: local-lvm
|
||||
thinpool data
|
||||
vgname pve
|
||||
content rootdir,images
|
||||
|
||||
===== ZFS POOLS (zpool list -v) =====
|
||||
|
||||
no pools available
|
||||
|
||||
===== FILESYSTEMS (df -h, local fs only) =====
|
||||
|
||||
Filesystem Type Size Used Avail Use% Mounted on
|
||||
/dev/mapper/pve-root ext4 94G 6.8G 83G 8% /
|
||||
/dev/fuse fuse 128M 24K 128M 1% /etc/pve
|
||||
|
||||
===== MOUNTED NFS / CIFS =====
|
||||
|
||||
|
||||
===== BACKUP JOBS (/etc/pve/jobs.cfg + /etc/vzdump.conf) =====
|
||||
|
||||
(/etc/pve/jobs.cfg not readable (no scheduled backups?))
|
||||
|
||||
----- vzdump defaults -----
|
||||
|
||||
===== BACKUP COVERAGE ANALYSIS =====
|
||||
|
||||
(/etc/pve/jobs.cfg not readable — no coverage data)
|
||||
|
||||
----- per-guest coverage -----
|
||||
ID VERDICT REASON
|
||||
100 NO no job targets this id
|
||||
|
||||
===== RECENT BACKUPS ON LOCAL DUMP STORAGES =====
|
||||
|
||||
|
||||
----- /var/lib/vz/dump/ -----
|
||||
|
||||
===== PBS TARGETS =====
|
||||
|
||||
(no PBS storage configured on this node)
|
||||
|
||||
===== REPLICATION JOBS (pvesr status) =====
|
||||
|
||||
JobID Enabled Target LastSync NextSync Duration FailCount State
|
||||
|
||||
===== LISTENING PORTS =====
|
||||
|
||||
0.0.0.0:111
|
||||
0.0.0.0:22
|
||||
[::]:111
|
||||
[::1]:25
|
||||
127.0.0.1:25
|
||||
127.0.0.1:85
|
||||
[::]:22
|
||||
*:3128
|
||||
*:8006
|
||||
|
||||
===== SUBSCRIPTION / UPDATES =====
|
||||
|
||||
status: notfound
|
||||
|
||||
----- apt upgradable (top 20) -----
|
||||
base-files/oldstable 12.4+deb12u13 amd64 [upgradable from: 12.4+deb12u5]
|
||||
bash/oldstable 5.2.15-2+b10 amd64 [upgradable from: 5.2.15-2+b2]
|
||||
bind9-dnsutils/oldstable-security 1:9.18.47-1~deb12u1 amd64 [upgradable from: 1:9.18.24-1]
|
||||
bind9-host/oldstable-security 1:9.18.47-1~deb12u1 amd64 [upgradable from: 1:9.18.24-1]
|
||||
bind9-libs/oldstable-security 1:9.18.47-1~deb12u1 amd64 [upgradable from: 1:9.18.24-1]
|
||||
bsdextrautils/oldstable 2.38.1-5+deb12u3 amd64 [upgradable from: 2.38.1-5+deb12u1]
|
||||
bsdutils/oldstable 1:2.38.1-5+deb12u3 amd64 [upgradable from: 1:2.38.1-5+deb12u1]
|
||||
btrfs-progs/oldstable 6.2-1+deb12u2 amd64 [upgradable from: 6.2-1]
|
||||
busybox/oldstable 1:1.35.0-4+b7 amd64 [upgradable from: 1:1.35.0-4+b3]
|
||||
ca-certificates/oldstable,oldstable-updates 20230311+deb12u1 all [upgradable from: 20230311]
|
||||
ceph-common/oldstable,oldstable-security 16.2.15+ds-0+deb12u1 amd64 [upgradable from: 16.2.11+ds-2]
|
||||
ceph-fuse/oldstable,oldstable-security 16.2.15+ds-0+deb12u1 amd64 [upgradable from: 16.2.11+ds-2]
|
||||
corosync/stable 3.1.10-pve2~bpo12+1 amd64 [upgradable from: 3.1.7-pve3]
|
||||
criu/oldstable 3.17.1-2+deb12u2 amd64 [upgradable from: 3.17.1-2]
|
||||
curl/oldstable 7.88.1-10+deb12u14 amd64 [upgradable from: 7.88.1-10+deb12u5]
|
||||
debian-archive-keyring/oldstable 2023.3+deb12u2 all [upgradable from: 2023.3+deb12u1]
|
||||
dirmngr/oldstable 2.2.40-1.1+deb12u2 amd64 [upgradable from: 2.2.40-1.1]
|
||||
distro-info-data/oldstable 0.58+deb12u6 all [upgradable from: 0.58+deb12u1]
|
||||
e2fsprogs/oldstable 1.47.0-2+b2 amd64 [upgradable from: 1.47.0-2]
|
||||
eject/oldstable 2.38.1-5+deb12u3 amd64 [upgradable from: 2.38.1-5+deb12u1]
|
||||
|
||||
===== DONE =====
|
||||
|
||||
Review this output for backup gaps (VERDICT=NO in coverage section)
|
||||
and anomalies in storage / PBS / replication sections.
|
||||
@@ -0,0 +1,80 @@
|
||||
|
||||
===== HOST =====
|
||||
|
||||
Hostname: sfsrv-ana.surefire.com
|
||||
Date: 2026-04-23T00:52:46-07:00
|
||||
Uptime: up 4 weeks, 9 hours, 12 minutes
|
||||
OS: Debian GNU/Linux 12 (bookworm)
|
||||
Kernel: 6.8.4-3-pve
|
||||
Arch: x86_64
|
||||
|
||||
===== HARDWARE =====
|
||||
|
||||
CPU cores: 24
|
||||
CPU model: Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
|
||||
MemTotal: 110.0 GB
|
||||
MemAvailable: 104.1 GB
|
||||
|
||||
===== GPUS =====
|
||||
|
||||
nvidia-smi not present (no NVIDIA GPUs or driver not installed)
|
||||
|
||||
===== FILESYSTEMS (df) =====
|
||||
|
||||
Filesystem Size Used Avail Use% Mounted on
|
||||
/dev/mapper/pve-root 94G 6.8G 83G 8% /
|
||||
/dev/fuse 128M 24K 128M 1% /etc/pve
|
||||
|
||||
===== PERSISTENT MOUNTS (/etc/fstab, non-comment) =====
|
||||
|
||||
/dev/pve/root / ext4 errors=remount-ro 0 1
|
||||
/dev/pve/swap none swap sw 0 0
|
||||
proc /proc proc defaults 0 0
|
||||
|
||||
===== TARGETED DATA PATHS =====
|
||||
|
||||
/opt (total: 4.0K)
|
||||
total 8
|
||||
drwxr-xr-x 2 root root 4096 2023-03-22 07:38 .
|
||||
drwxr-xr-x 18 root root 4096 2024-05-08 19:29 ..
|
||||
|
||||
/srv (total: 4.0K)
|
||||
total 8
|
||||
drwxr-xr-x 2 root root 4096 2023-03-22 07:38 .
|
||||
drwxr-xr-x 18 root root 4096 2024-05-08 19:29 ..
|
||||
|
||||
|
||||
===== DOCKER =====
|
||||
|
||||
docker not installed
|
||||
|
||||
===== COMPOSE FILES (/opt/docker/compose/) =====
|
||||
|
||||
/opt/docker/compose not present
|
||||
|
||||
===== CONFIG LAYOUT (/opt/docker/conf/ — top 200 entries) =====
|
||||
|
||||
/opt/docker/conf not present
|
||||
|
||||
===== LISTENING PORTS =====
|
||||
|
||||
0.0.0.0:111
|
||||
0.0.0.0:22
|
||||
[::]:111
|
||||
[::1]:25
|
||||
127.0.0.1:25
|
||||
127.0.0.1:85
|
||||
[::]:22
|
||||
*:3128
|
||||
*:8006
|
||||
|
||||
===== MODEL / HUGGINGFACE CACHES =====
|
||||
|
||||
|
||||
===== DOCKER-ADJACENT SYSTEMD SERVICES =====
|
||||
|
||||
(none matching)
|
||||
|
||||
===== DONE =====
|
||||
|
||||
Paste the above back into the chat, or pass a path as argv[1] to save.
|
||||
Reference in New Issue
Block a user