Exclude an external shelf by explicit config, not by path shape

Testing on a second host showed the internal/external split cannot be
inferred: iz-pve1's MD1200 sits behind its own HBA while iz-pve0's
internal drives sit behind a SAS expander, so the previous
"pci-*-scsi-*" glob matched every drive on one host and none on the
other. DISK_EXCLUDE_PATTERN now names the shelf's HBA per host.

Also fixes parse_disk_limit, which used awk's for-in over split fields
and so returned 0 or 70 from "0/70" depending on iteration order.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Zeb Hering
2026-08-28 18:43:20 -07:00
parent 562d8e8d46
commit 64e6e03dba
3 changed files with 34 additions and 8 deletions

View File

@@ -127,6 +127,7 @@ systemctl enable --now fan_speed.service
| `HDD_LIMIT_CAP` / `SSD_LIMIT_CAP` | `60` / `70` | Ceiling on what a drive may claim | | `HDD_LIMIT_CAP` / `SSD_LIMIT_CAP` | `60` / `70` | Ceiling on what a drive may claim |
| `TREND_SAMPLES` | `90` | Window length, in passes | | `TREND_SAMPLES` | `90` | Window length, in passes |
| `TREND_RISE_ALARM` | `8` | Degrees of climb that alarms | | `TREND_RISE_ALARM` | `8` | Degrees of climb that alarms |
| `DISK_EXCLUDE_PATTERN` | *(empty)* | by-path substring to skip, e.g. an external shelf's HBA |
| `ALERT_EMAIL` | `Servers@ntfy1.izebra.xyz` | | | `ALERT_EMAIL` | `Servers@ntfy1.izebra.xyz` | |
| `ALERT_COOLDOWN` | `3600` | Seconds between repeats of one alarm | | `ALERT_COOLDOWN` | `3600` | Seconds between repeats of one alarm |
| `SMART_CHECK_INTERVAL` | `3600` | Seconds between SMART sweeps | | `SMART_CHECK_INTERVAL` | `3600` | Seconds between SMART sweeps |
@@ -145,8 +146,12 @@ trusting them anywhere else.
- Kernel device names are not stable — a shelf rescan renamed `sdaa``sdai` to - Kernel device names are not stable — a shelf rescan renamed `sdaa``sdai` to
`sds``sdaa` mid-session. Everything resolves through `/dev/disk/by-path` on every `sds``sdaa` mid-session. Everything resolves through `/dev/disk/by-path` on every
pass; never persist an `sdX`. pass; never persist an `sdX`.
- `DISK_GLOB` matches internal drives only (`pci-*-scsi-*`). Drives behind a SAS expander - **An external shelf must be excluded by hand.** Set `DISK_EXCLUDE_PATTERN` to its HBA's
are a separate enclosure with separate cooling and are deliberately excluded. PCI address in the systemd unit. This cannot be inferred: `iz-pve1`'s MD1200 sits behind
its own HBA at `pci-0000:04:00.0`, while `iz-pve0`'s *internal* drives sit behind a SAS
expander at `pci-0000:02:00.0` — so "behind an expander" identifies an external
enclosure on one host and the internal backplane on the other. `iz-pve1` sets it,
`iz-pve0` has no shelf and leaves it empty.
- Drive temperature limits are read once at startup. They do not change, and `smartctl -x` - Drive temperature limits are read once at startup. They do not change, and `smartctl -x`
is far heavier than the `-A` used on the fast loop. is far heavier than the `-A` used on the fast loop.
- `smartctl -n standby` throughout, so a sleeping drive is skipped rather than spun up - `smartctl -n standby` throughout, so a sleeping drive is skipped rather than spun up

View File

@@ -2,7 +2,14 @@
Description=Fan_Speed service Description=Fan_Speed service
[Service] [Service]
ExecStart=/bin/bash -c /root/fan_speed/fan_speed.sh # Hosts with an external disk shelf must exclude that HBA - the shelf has its
# own cooling and must not drive server fans. Find the PCI address with
# `ls /dev/disk/by-path/`; there is no way to infer it (see monitor.sh).
# iz-pve1 needs this; iz-pve0 has no shelf and needs nothing.
#Environment=DISK_EXCLUDE_PATTERN=pci-0000:04:00.0
ExecStart=/root/fan_speed/fan_speed.sh
Restart=always
RestartSec=10
[Install] [Install]
WantedBy=multi-user.target WantedBy=multi-user.target

View File

@@ -8,9 +8,17 @@
# ---------------------------------------------------------------- configuration # ---------------------------------------------------------------- configuration
# Internal drives only. The DAS shelf enumerates behind a SAS expander DISK_GLOB=${DISK_GLOB:-/dev/disk/by-path/*}
# (*-sas-exp*) and has its own controller - it must not drive server fans.
DISK_GLOB=${DISK_GLOB:-/dev/disk/by-path/pci-*-scsi-*} # Drives to leave alone, matched against the by-path link. Set this to the PCI
# address of an external shelf's HBA on any host that has one.
#
# There is no reliable way to infer this. An external enclosure looks exactly
# like an internal backplane from /dev/disk/by-path: iz-pve1's MD1200 sits
# behind its own HBA at pci-0000:04:00.0, while iz-pve0's *internal* drives sit
# behind a SAS expander at pci-0000:02:00.0. Guessing from "-sas-exp" picks up
# the wrong set on one host or the other, so it has to be stated per host.
DISK_EXCLUDE_PATTERN=${DISK_EXCLUDE_PATTERN:-}
# Each drive's fan ramp is derived from its own maximum operating temperature: # Each drive's fan ramp is derived from its own maximum operating temperature:
# ramp starts at limit - DISK_RAMP_LOW_OFFSET # ramp starts at limit - DISK_RAMP_LOW_OFFSET
@@ -53,6 +61,9 @@ disk_devices() {
local link dev local link dev
for link in $DISK_GLOB; do for link in $DISK_GLOB; do
case "$link" in *-part*) continue ;; esac case "$link" in *-part*) continue ;; esac
if [ -n "$DISK_EXCLUDE_PATTERN" ]; then
case "$link" in *"$DISK_EXCLUDE_PATTERN"*) continue ;; esac
fi
[ -e "$link" ] || continue [ -e "$link" ] || continue
dev=$(readlink -f "$link") dev=$(readlink -f "$link")
[ -b "$dev" ] && echo "$dev" [ -b "$dev" ] && echo "$dev"
@@ -73,7 +84,10 @@ parse_disk_temperature() {
# smartctl -x output on stdin -> the drive's own maximum operating temperature. # smartctl -x output on stdin -> the drive's own maximum operating temperature.
parse_disk_limit() { parse_disk_limit() {
awk ' awk '
/Min\/Max Temperature Limit/ { split($0, a, /[ \/]+/); for (i in a) if (a[i] ~ /^[0-9]+$/) v = a[i]; print v; exit } /Min\/Max Temperature Limit/ {
if (match($0, /-?[0-9]+\/[0-9]+/)) { split(substr($0, RSTART, RLENGTH), a, "/"); print a[2] }
exit
}
/Drive Trip Temperature/ { print $4; exit }' /Drive Trip Temperature/ { print $4; exit }'
} }