Enforce that full cooling arrives no later than the disk alarm
DISK_RAMP_HIGH_OFFSET below DISK_ALARM_OFFSET means the alarm fires while there is still cooling left unused. iz-pve0 was tuned that way by hand (full at limit-2, alarm at limit-5), which would have mailed 3c before the fans were actually flat out, for every drive class. check_ramp_ordering() now clamps at startup and logs it, rather than exiting - a fan controller that refuses to start leaves the fans wherever they were. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
18
README.md
18
README.md
@@ -57,6 +57,24 @@ preheated exhaust air. A shared threshold would peg the fans for drives that are
|
|||||||
and ignore the ones that are not. Measuring each drive against its own envelope makes
|
and ignore the ones that are not. Measuring each drive against its own envelope makes
|
||||||
the curve independent of both drive technology and bay position.
|
the curve independent of both drive technology and bay position.
|
||||||
|
|
||||||
|
A drive with a lower tolerance drives the fans even while a hotter drive does not.
|
||||||
|
With `iz-pve0`'s settings (floor 28%, ramp `limit-8` to `limit-5`):
|
||||||
|
|
||||||
|
| Drive | Limit | Ramp | Asks at 45c | at 50c | at 55c |
|
||||||
|
|---|---|---|---|---|---|
|
||||||
|
| Samsung 860 EVO | 70c | 62-65c | 28% | 28% | 28% |
|
||||||
|
| Toshiba / Seagate HDD | 60c | 52-55c | 28% | 28% | **50%** |
|
||||||
|
| a 55c-rated drive | 55c | 47-50c | 28% | **50%** | **50%** |
|
||||||
|
|
||||||
|
At 55c the SSD contributes nothing while the spinner asks for full speed, which is the
|
||||||
|
whole point: the same temperature means different things to different drives.
|
||||||
|
|
||||||
|
**Full cooling must arrive no later than the alarm.** `DISK_RAMP_HIGH_OFFSET` must be
|
||||||
|
`>=` `DISK_ALARM_OFFSET`, or the alarm mails you while there is still cooling left unused.
|
||||||
|
`check_ramp_ordering()` clamps this at startup and logs when it does — it clamps rather
|
||||||
|
than exits, because a fan controller that refuses to start leaves the fans wherever they
|
||||||
|
happened to be.
|
||||||
|
|
||||||
**Reported limits are clamped by class.** They are not uniformly trustworthy — Samsung
|
**Reported limits are clamped by class.** They are not uniformly trustworthy — Samsung
|
||||||
and Kioxia report a real operating maximum (70), Toshiba and Seagate report 60, and WD
|
and Kioxia report a real operating maximum (70), Toshiba and Seagate report 60, and WD
|
||||||
Reds report **85**, which is the SCT critical limit and not somewhere you want a drive
|
Reds report **85**, which is the SCT critical limit and not somewhere you want a drive
|
||||||
|
|||||||
@@ -117,6 +117,11 @@ selftest() {
|
|||||||
# leak is what let the setpoint drift 32 -> 36 -> 31 on iz-pve0.
|
# leak is what let the setpoint drift 32 -> 36 -> 31 on iz-pve0.
|
||||||
check "re-assert re-pushes held value" 32 "$(next_fan_speed 34 32)"
|
check "re-assert re-pushes held value" 32 "$(next_fan_speed 34 32)"
|
||||||
|
|
||||||
|
echo "ramp/alarm ordering (full cooling must arrive no later than the alarm):"
|
||||||
|
ramp_ordering_ok 8 5 && check "full at limit-8, alarm at limit-5" ok ok || check "full at limit-8, alarm at limit-5" ok bad
|
||||||
|
ramp_ordering_ok 5 5 && check "full and alarm coincide" ok ok || check "full and alarm coincide" ok bad
|
||||||
|
ramp_ordering_ok 2 5 && check "full at limit-2 is rejected" bad ok || check "full at limit-2 is rejected" bad bad
|
||||||
|
|
||||||
echo "alert rate limiting:"
|
echo "alert rate limiting:"
|
||||||
local out
|
local out
|
||||||
out=$( { raise_alert testkey "first" "body"; raise_alert testkey "second" "body"; } | grep -c '^MAIL\[' )
|
out=$( { raise_alert testkey "first" "body"; raise_alert testkey "second" "body"; } | grep -c '^MAIL\[' )
|
||||||
@@ -210,6 +215,7 @@ fi
|
|||||||
#######################################
|
#######################################
|
||||||
# Read every drive's own temperature limit once - they do not change, and
|
# Read every drive's own temperature limit once - they do not change, and
|
||||||
# smartctl -x is far heavier than the -A used on the fast loop.
|
# smartctl -x is far heavier than the -A used on the fast loop.
|
||||||
|
check_ramp_ordering
|
||||||
cache_disk_limits
|
cache_disk_limits
|
||||||
echo "Monitoring $DISK_COUNT_EXPECTED disk(s)."
|
echo "Monitoring $DISK_COUNT_EXPECTED disk(s)."
|
||||||
LAST_SMART_CHECK=0
|
LAST_SMART_CHECK=0
|
||||||
|
|||||||
21
monitor.sh
21
monitor.sh
@@ -29,8 +29,29 @@ DISK_RAMP_HIGH_OFFSET=${DISK_RAMP_HIGH_OFFSET:-8}
|
|||||||
|
|
||||||
# A drive this close to its own limit raises an alarm - the fans are already
|
# A drive this close to its own limit raises an alarm - the fans are already
|
||||||
# flat out for it and it is still climbing.
|
# flat out for it and it is still climbing.
|
||||||
|
#
|
||||||
|
# That sentence is only true if full fan speed is reached at or before the alarm
|
||||||
|
# point, i.e. DISK_RAMP_HIGH_OFFSET >= DISK_ALARM_OFFSET. Set the other way round
|
||||||
|
# the alarm mails you while there is still cooling left unused, which is exactly
|
||||||
|
# backwards. check_ramp_ordering() enforces it at startup.
|
||||||
DISK_ALARM_OFFSET=${DISK_ALARM_OFFSET:-5}
|
DISK_ALARM_OFFSET=${DISK_ALARM_OFFSET:-5}
|
||||||
|
|
||||||
|
# True when full cooling is reached no later than the alarm point.
|
||||||
|
ramp_ordering_ok() {
|
||||||
|
[ "${1:-$DISK_RAMP_HIGH_OFFSET}" -ge "${2:-$DISK_ALARM_OFFSET}" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
# Clamp rather than refuse to start - a fan controller that exits leaves the
|
||||||
|
# fans wherever they were, which is worse than a slightly wrong curve.
|
||||||
|
check_ramp_ordering() {
|
||||||
|
if ! ramp_ordering_ok; then
|
||||||
|
echo "/!\\ DISK_RAMP_HIGH_OFFSET ($DISK_RAMP_HIGH_OFFSET) is below DISK_ALARM_OFFSET ($DISK_ALARM_OFFSET):" >&2
|
||||||
|
echo " the alarm would fire before the fans reach full speed. Raising it to $DISK_ALARM_OFFSET." >&2
|
||||||
|
log_line "config: raised DISK_RAMP_HIGH_OFFSET $DISK_RAMP_HIGH_OFFSET -> $DISK_ALARM_OFFSET"
|
||||||
|
DISK_RAMP_HIGH_OFFSET=$DISK_ALARM_OFFSET
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
# Drives report their limit but do not agree on what it means: Samsung and
|
# Drives report their limit but do not agree on what it means: Samsung and
|
||||||
# Kioxia report a true operating maximum (70), Toshiba and Seagate report 60,
|
# Kioxia report a true operating maximum (70), Toshiba and Seagate report 60,
|
||||||
# WD Reds report 85 - the SCT critical limit, not an operating maximum.
|
# WD Reds report 85 - the SCT critical limit, not an operating maximum.
|
||||||
|
|||||||
Reference in New Issue
Block a user