From 64e6e03dba239c2f77fc8407ba19351e631df317 Mon Sep 17 00:00:00 2001 From: Zeb Hering Date: Fri, 28 Aug 2026 18:43:20 -0700 Subject: [PATCH] Exclude an external shelf by explicit config, not by path shape Testing on a second host showed the internal/external split cannot be inferred: iz-pve1's MD1200 sits behind its own HBA while iz-pve0's internal drives sit behind a SAS expander, so the previous "pci-*-scsi-*" glob matched every drive on one host and none on the other. DISK_EXCLUDE_PATTERN now names the shelf's HBA per host. Also fixes parse_disk_limit, which used awk's for-in over split fields and so returned 0 or 70 from "0/70" depending on iteration order. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 9 +++++++-- fan_speed.service | 9 ++++++++- monitor.sh | 24 +++++++++++++++++++----- 3 files changed, 34 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index f0305a0..618a1b1 100644 --- a/README.md +++ b/README.md @@ -127,6 +127,7 @@ systemctl enable --now fan_speed.service | `HDD_LIMIT_CAP` / `SSD_LIMIT_CAP` | `60` / `70` | Ceiling on what a drive may claim | | `TREND_SAMPLES` | `90` | Window length, in passes | | `TREND_RISE_ALARM` | `8` | Degrees of climb that alarms | +| `DISK_EXCLUDE_PATTERN` | *(empty)* | by-path substring to skip, e.g. an external shelf's HBA | | `ALERT_EMAIL` | `Servers@ntfy1.izebra.xyz` | | | `ALERT_COOLDOWN` | `3600` | Seconds between repeats of one alarm | | `SMART_CHECK_INTERVAL` | `3600` | Seconds between SMART sweeps | @@ -145,8 +146,12 @@ trusting them anywhere else. - Kernel device names are not stable — a shelf rescan renamed `sdaa`–`sdai` to `sds`–`sdaa` mid-session. Everything resolves through `/dev/disk/by-path` on every pass; never persist an `sdX`. -- `DISK_GLOB` matches internal drives only (`pci-*-scsi-*`). Drives behind a SAS expander - are a separate enclosure with separate cooling and are deliberately excluded. +- **An external shelf must be excluded by hand.** Set `DISK_EXCLUDE_PATTERN` to its HBA's + PCI address in the systemd unit. This cannot be inferred: `iz-pve1`'s MD1200 sits behind + its own HBA at `pci-0000:04:00.0`, while `iz-pve0`'s *internal* drives sit behind a SAS + expander at `pci-0000:02:00.0` — so "behind an expander" identifies an external + enclosure on one host and the internal backplane on the other. `iz-pve1` sets it, + `iz-pve0` has no shelf and leaves it empty. - Drive temperature limits are read once at startup. They do not change, and `smartctl -x` is far heavier than the `-A` used on the fast loop. - `smartctl -n standby` throughout, so a sleeping drive is skipped rather than spun up diff --git a/fan_speed.service b/fan_speed.service index fd3b715..db0cd34 100644 --- a/fan_speed.service +++ b/fan_speed.service @@ -2,7 +2,14 @@ Description=Fan_Speed service [Service] -ExecStart=/bin/bash -c /root/fan_speed/fan_speed.sh +# Hosts with an external disk shelf must exclude that HBA - the shelf has its +# own cooling and must not drive server fans. Find the PCI address with +# `ls /dev/disk/by-path/`; there is no way to infer it (see monitor.sh). +# iz-pve1 needs this; iz-pve0 has no shelf and needs nothing. +#Environment=DISK_EXCLUDE_PATTERN=pci-0000:04:00.0 +ExecStart=/root/fan_speed/fan_speed.sh +Restart=always +RestartSec=10 [Install] WantedBy=multi-user.target diff --git a/monitor.sh b/monitor.sh index 5abb2b7..e0bda3d 100644 --- a/monitor.sh +++ b/monitor.sh @@ -8,9 +8,17 @@ # ---------------------------------------------------------------- configuration -# Internal drives only. The DAS shelf enumerates behind a SAS expander -# (*-sas-exp*) and has its own controller - it must not drive server fans. -DISK_GLOB=${DISK_GLOB:-/dev/disk/by-path/pci-*-scsi-*} +DISK_GLOB=${DISK_GLOB:-/dev/disk/by-path/*} + +# Drives to leave alone, matched against the by-path link. Set this to the PCI +# address of an external shelf's HBA on any host that has one. +# +# There is no reliable way to infer this. An external enclosure looks exactly +# like an internal backplane from /dev/disk/by-path: iz-pve1's MD1200 sits +# behind its own HBA at pci-0000:04:00.0, while iz-pve0's *internal* drives sit +# behind a SAS expander at pci-0000:02:00.0. Guessing from "-sas-exp" picks up +# the wrong set on one host or the other, so it has to be stated per host. +DISK_EXCLUDE_PATTERN=${DISK_EXCLUDE_PATTERN:-} # Each drive's fan ramp is derived from its own maximum operating temperature: # ramp starts at limit - DISK_RAMP_LOW_OFFSET @@ -53,6 +61,9 @@ disk_devices() { local link dev for link in $DISK_GLOB; do case "$link" in *-part*) continue ;; esac + if [ -n "$DISK_EXCLUDE_PATTERN" ]; then + case "$link" in *"$DISK_EXCLUDE_PATTERN"*) continue ;; esac + fi [ -e "$link" ] || continue dev=$(readlink -f "$link") [ -b "$dev" ] && echo "$dev" @@ -73,8 +84,11 @@ parse_disk_temperature() { # smartctl -x output on stdin -> the drive's own maximum operating temperature. parse_disk_limit() { awk ' - /Min\/Max Temperature Limit/ { split($0, a, /[ \/]+/); for (i in a) if (a[i] ~ /^[0-9]+$/) v = a[i]; print v; exit } - /Drive Trip Temperature/ { print $4; exit }' + /Min\/Max Temperature Limit/ { + if (match($0, /-?[0-9]+\/[0-9]+/)) { split(substr($0, RSTART, RLENGTH), a, "/"); print a[2] } + exit + } + /Drive Trip Temperature/ { print $4; exit }' } # -n standby: skip a sleeping drive rather than spinning it up to measure it.