Calibrate the CPU ramp per host and fix the deadband leak

Two problems found by A/B against Dell's own profile on an R720xd, 8
minute windows at matched CPU temperature:

The CPU ramp started at a fixed 45c, calibrated on a chassis idling at
48c. On one idling at 58c that sat 40% up the ramp doing nothing useful
and ran 6878 RPM against stock's 6240. The ramp is now derived from the
CPU sensor's own upper-non-critical threshold, and the fan floor is
per-host since PWM->RPM is chassis specific.

The 5 minute re-assert re-pushed the current request rather than the
held value, so the deadband leaked a few percent every interval and the
setpoint drifted 32 -> 36 -> 31. next_fan_speed now separates "has it
moved enough to adopt" from "is it time to re-push".

Also documents that running above stock is correct on hosts whose drives
need it: removing the drives' vote on iz-pve0 hit stock RPM exactly and
took two rear-bay SSDs from 54c to 61c in minutes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Zeb Hering
2026-08-28 21:12:54 -07:00
parent 56ae7542a2
commit 1be394a0ac
3 changed files with 113 additions and 43 deletions

View File

@@ -52,20 +52,30 @@ FAN_SPEED_DEADBAND=${FAN_SPEED_DEADBAND:-5}
# ipmitool call every 5 minutes is cheaper than finding out the hard way.
FAN_REASSERT_INTERVAL=${FAN_REASSERT_INTERVAL:-300}
# should_apply_fan_speed <wanted> <currently applied> <epoch it was applied>
# True when the change is worth making. A request for full speed is never held
# back, and the first pass always applies.
should_apply_fan_speed() {
local wanted=$1 applied=$2 applied_at=$3 delta now
[ -n "$applied" ] || return 0
# fan_speed_changed_enough <wanted> <currently applied>
# True when the request has moved far enough to be worth acting on. A request
# for full speed is never held back.
fan_speed_changed_enough() {
local wanted=$1 applied=$2 delta
[ "$wanted" -ge "$HIGH_FAN_SPEED" ] && return 0
now=$(date +%s)
[ $((now - applied_at)) -ge "$FAN_REASSERT_INTERVAL" ] && return 0
delta=$((wanted - applied))
[ "$delta" -lt 0 ] && delta=$((-delta))
[ "$delta" -ge "$FAN_SPEED_DEADBAND" ]
}
# next_fan_speed <wanted> <currently applied> -> the speed to hold from here.
# Kept separate from the re-assert timer on purpose: a re-assert must re-push
# the value already being held, never silently adopt the current request, or
# the deadband leaks a few percent every FAN_REASSERT_INTERVAL.
next_fan_speed() {
local wanted=$1 applied=$2
if [ -z "$applied" ] || fan_speed_changed_enough "$wanted" "$applied"; then
echo "$wanted"
else
echo "$applied"
fi
}
ALERT_EMAIL=${ALERT_EMAIL:-Servers@ntfy1.izebra.xyz}
ALERT_COOLDOWN=${ALERT_COOLDOWN:-3600}
STATE_DIR=${STATE_DIR:-/root/fan_speed/state}