Enforce that full cooling arrives no later than the disk alarm

DISK_RAMP_HIGH_OFFSET below DISK_ALARM_OFFSET means the alarm fires
while there is still cooling left unused. iz-pve0 was tuned that way by
hand (full at limit-2, alarm at limit-5), which would have mailed 3c
before the fans were actually flat out, for every drive class.

check_ramp_ordering() now clamps at startup and logs it, rather than
exiting - a fan controller that refuses to start leaves the fans
wherever they were.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Zeb Hering
2026-08-28 21:49:50 -07:00
parent 02cd1a4004
commit 86f3b0c323
3 changed files with 45 additions and 0 deletions

View File

@@ -57,6 +57,24 @@ preheated exhaust air. A shared threshold would peg the fans for drives that are
and ignore the ones that are not. Measuring each drive against its own envelope makes and ignore the ones that are not. Measuring each drive against its own envelope makes
the curve independent of both drive technology and bay position. the curve independent of both drive technology and bay position.
A drive with a lower tolerance drives the fans even while a hotter drive does not.
With `iz-pve0`'s settings (floor 28%, ramp `limit-8` to `limit-5`):
| Drive | Limit | Ramp | Asks at 45c | at 50c | at 55c |
|---|---|---|---|---|---|
| Samsung 860 EVO | 70c | 62-65c | 28% | 28% | 28% |
| Toshiba / Seagate HDD | 60c | 52-55c | 28% | 28% | **50%** |
| a 55c-rated drive | 55c | 47-50c | 28% | **50%** | **50%** |
At 55c the SSD contributes nothing while the spinner asks for full speed, which is the
whole point: the same temperature means different things to different drives.
**Full cooling must arrive no later than the alarm.** `DISK_RAMP_HIGH_OFFSET` must be
`>=` `DISK_ALARM_OFFSET`, or the alarm mails you while there is still cooling left unused.
`check_ramp_ordering()` clamps this at startup and logs when it does — it clamps rather
than exits, because a fan controller that refuses to start leaves the fans wherever they
happened to be.
**Reported limits are clamped by class.** They are not uniformly trustworthy — Samsung **Reported limits are clamped by class.** They are not uniformly trustworthy — Samsung
and Kioxia report a real operating maximum (70), Toshiba and Seagate report 60, and WD and Kioxia report a real operating maximum (70), Toshiba and Seagate report 60, and WD
Reds report **85**, which is the SCT critical limit and not somewhere you want a drive Reds report **85**, which is the SCT critical limit and not somewhere you want a drive

View File

@@ -117,6 +117,11 @@ selftest() {
# leak is what let the setpoint drift 32 -> 36 -> 31 on iz-pve0. # leak is what let the setpoint drift 32 -> 36 -> 31 on iz-pve0.
check "re-assert re-pushes held value" 32 "$(next_fan_speed 34 32)" check "re-assert re-pushes held value" 32 "$(next_fan_speed 34 32)"
echo "ramp/alarm ordering (full cooling must arrive no later than the alarm):"
ramp_ordering_ok 8 5 && check "full at limit-8, alarm at limit-5" ok ok || check "full at limit-8, alarm at limit-5" ok bad
ramp_ordering_ok 5 5 && check "full and alarm coincide" ok ok || check "full and alarm coincide" ok bad
ramp_ordering_ok 2 5 && check "full at limit-2 is rejected" bad ok || check "full at limit-2 is rejected" bad bad
echo "alert rate limiting:" echo "alert rate limiting:"
local out local out
out=$( { raise_alert testkey "first" "body"; raise_alert testkey "second" "body"; } | grep -c '^MAIL\[' ) out=$( { raise_alert testkey "first" "body"; raise_alert testkey "second" "body"; } | grep -c '^MAIL\[' )
@@ -210,6 +215,7 @@ fi
####################################### #######################################
# Read every drive's own temperature limit once - they do not change, and # Read every drive's own temperature limit once - they do not change, and
# smartctl -x is far heavier than the -A used on the fast loop. # smartctl -x is far heavier than the -A used on the fast loop.
check_ramp_ordering
cache_disk_limits cache_disk_limits
echo "Monitoring $DISK_COUNT_EXPECTED disk(s)." echo "Monitoring $DISK_COUNT_EXPECTED disk(s)."
LAST_SMART_CHECK=0 LAST_SMART_CHECK=0

View File

@@ -29,8 +29,29 @@ DISK_RAMP_HIGH_OFFSET=${DISK_RAMP_HIGH_OFFSET:-8}
# A drive this close to its own limit raises an alarm - the fans are already # A drive this close to its own limit raises an alarm - the fans are already
# flat out for it and it is still climbing. # flat out for it and it is still climbing.
#
# That sentence is only true if full fan speed is reached at or before the alarm
# point, i.e. DISK_RAMP_HIGH_OFFSET >= DISK_ALARM_OFFSET. Set the other way round
# the alarm mails you while there is still cooling left unused, which is exactly
# backwards. check_ramp_ordering() enforces it at startup.
DISK_ALARM_OFFSET=${DISK_ALARM_OFFSET:-5} DISK_ALARM_OFFSET=${DISK_ALARM_OFFSET:-5}
# True when full cooling is reached no later than the alarm point.
ramp_ordering_ok() {
[ "${1:-$DISK_RAMP_HIGH_OFFSET}" -ge "${2:-$DISK_ALARM_OFFSET}" ]
}
# Clamp rather than refuse to start - a fan controller that exits leaves the
# fans wherever they were, which is worse than a slightly wrong curve.
check_ramp_ordering() {
if ! ramp_ordering_ok; then
echo "/!\\ DISK_RAMP_HIGH_OFFSET ($DISK_RAMP_HIGH_OFFSET) is below DISK_ALARM_OFFSET ($DISK_ALARM_OFFSET):" >&2
echo " the alarm would fire before the fans reach full speed. Raising it to $DISK_ALARM_OFFSET." >&2
log_line "config: raised DISK_RAMP_HIGH_OFFSET $DISK_RAMP_HIGH_OFFSET -> $DISK_ALARM_OFFSET"
DISK_RAMP_HIGH_OFFSET=$DISK_ALARM_OFFSET
fi
}
# Drives report their limit but do not agree on what it means: Samsung and # Drives report their limit but do not agree on what it means: Samsung and
# Kioxia report a true operating maximum (70), Toshiba and Seagate report 60, # Kioxia report a true operating maximum (70), Toshiba and Seagate report 60,
# WD Reds report 85 - the SCT critical limit, not an operating maximum. # WD Reds report 85 - the SCT critical limit, not an operating maximum.