Files
md1200-fan-control/README.md
Zeb Hering 9961e972df Move the shelf ramp above the drives' working range, add a deadband
Retiring 8x2TB for 4x8TB + a 16TB moved the hottest drive to 40-41c,
exactly the old 40c ramp start, and the fan flipped 10<->15% every 30
seconds. The ramp is now 44-56c, which puts the drives back on the floor
with 19c of headroom to their 60c limit.

Adds the deadband from fan_speed.sh so the shelf does not chase every
degree once drives do climb into the ramp, with a downward-only periodic
settle so damping cannot leave it permanently loud.

selftest and disks no longer take the serial lock - they do not open the
port, and refusing to run them while the service is up is unhelpful.

The new tests caught a real bug: in `local a=$1 b=$a` bash expands the
right-hand sides before the locals exist, so the held value was silently
empty.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-30 16:52:12 -07:00

7.9 KiB
Raw Permalink Blame History

md1200-fan-control

Temperature-aware fan control for a Dell PowerVault MD1200 disk shelf, driven over the EMM's serial debug console.

Out of the box an MD1200 runs its fans at a fixed, very loud speed. The EMM firmware accepts a _shutup <percent> command that overrides it — but it forgets the setting after a few seconds, and it has no idea how hot your drives actually are. This repo reads drive temperatures on the host and re-sends an appropriate fan speed on a keepalive loop.

Hardware you need

The MD1200 EMM exposes its debug console on the top-left RJ45 port of the left EMM (the one labelled for service use, not the SAS ports). It is not Ethernet — it is RS-232 behind an RJ45 jack, so you need the Dell service cable:

Dell Password Reset / Service Cable — part MN657, also sold as CT109.

Same cable is listed for the MD1000 / MD1220 / MD3000 / MD3200 family, so search on any of those. It terminates in a DB9 serial connector; if your host has no physical serial port, add any USB-to-DB9 adapter and point the script at the resulting /dev/ttyUSB*. On the machine this was written for, the cable lands on an onboard UART at /dev/ttyS1.

Serial settings: 38400 8-N-1, no flow control.

Note: Dell's own KB articles list 115200 for MD3xxx/ME3xxx controllers. That is a different port on different hardware. The MD1200 EMM debug console is 38400, which is what this repo uses and what was verified against the real shelf.

Once connected you get a prompt like:

BlueDress.106.000 >

Debug commands

There is no help — the firmware answers unknown_cmd to help, _help and ?. The two commands this project relies on:

Command Effect
_shutup <0-100> Set fan speed to that percentage. Forgotten if not re-sent.
_temp_rd Print the enclosure temperature sensors.

_temp_rd output looks like:

  BP_1[2] = 30c      <- drive backplane
  BP_2[3] = 31c      <- drive backplane
  SIM0[0] = 36c      <- enclosure management module
  EXP0[4] = 59c      <- SAS expander chip, runs hot by design

  AVG = 39c

Commands are terminated with a bare \r (not \n).

What this does

das_fanctl.sh runs as a systemd service and loops:

  1. Read the temperature of every drive in the shelf with smartctl, take the hottest.
  2. Interpolate a fan percentage between the low and high thresholds.
  3. Send _shutup <percent>, and re-send it every 5s so the EMM doesn't revert.
  4. Re-read temperatures every 30s.

Drive temperature, not enclosure air. The BP_* sensors read backplane intake air, which sat at 30c while the drives themselves were 3239c — it lags what you actually care about. Drive temps come from smartctl -n standby, so a sleeping drive is skipped rather than spun up just to be measured. If no drive answers (all in standby, smartctl missing), it falls back to the hottest BP_* sensor with its own threshold pair.

EXP0 is deliberately ignored. The SAS expander idles around 59c by design; including it in the curve pegs the fans permanently.

Usage

das_fanctl.sh            # the service loop (default)
das_fanctl.sh once       # one read + set, prints what it did
das_fanctl.sh disks      # per-drive temperatures and the hottest
das_fanctl.sh temps      # raw _temp_rd output from the shelf
das_fanctl.sh selftest   # asserts the parser, the curve, and that drives are readable

Install

The script sources functions.sh by absolute path, so it expects to live in /root/fan_speed/:

install -m 755 das_fanctl.sh functions.sh /root/fan_speed/
mkdir -p /root/fan_speed/log
install -m 644 fan_speed_das.service /etc/systemd/system/
systemctl daemon-reload
systemctl enable --now fan_speed_das.service

Check it: das_fanctl.sh selftest, then tail -f /root/fan_speed/log/das_fan.log.

2026-08-24 23:26:03 || DAS disk:39c | Fan Speed:10

Tuning

All knobs are at the top of das_fanctl.sh:

Variable Default Meaning
PORT /dev/ttyS1 Serial device the service cable lands on
CHECK_INTERVAL 30 Seconds between temperature readings
KEEPALIVE_INTERVAL 5 How often to re-send _shutup
DISK_LOW_TEMPERATURE_THRESHOLD 44 At/below this, run at LOW_FAN_SPEED
DISK_HIGH_TEMPERATURE_THRESHOLD 56 At/above this, run at HIGH_FAN_SPEED
FAN_SPEED_DEADBAND 8 Hold the current speed until the request moves this far
SETTLE_INTERVAL 300 Allow a downward settle this often
LOW_FAN_SPEED 10 Percent
HIGH_FAN_SPEED 60 Percent
BP_*_THRESHOLD 30 / 45 Fallback curve, backplane air sensors

Start the ramp above the drives' normal working range. This is the single most common way to make a shelf cycle audibly: if the ramp starts inside the range the drives already occupy, ordinary drift crosses it and the fans flip every cycle. The curve is ~4%/degC, so one degree moves the fan noticeably.

Worked example: the original 4050c was sized for 8×2TB drives idling at 3239c. Retiring those for 4×8TB + a 16TB moved the hottest drive to 4041c — exactly the old ramp start — and the fan flipped 10↔15% every 30 seconds (18 samples at 10, 22 at 15). Moving the ramp to 4456c put the drives back on the floor: 11 consecutive samples at 10%, and they still have 19c of headroom to their 60c limit.

FAN_SPEED_DEADBAND damps what remains, so once drives do climb into the ramp it does not chase every degree. A settle is permitted every SETTLE_INTERVAL, downward only — a symmetric deadband would catch the fan on the way up and never let it back down.

KEEPALIVE_INTERVAL is inherited from the 5s cadence the original setup used, not measured. How fast the EMM actually reverts is unknown; if the fans audibly surge between keepalives, lower it.

Notes

  • Only one process may talk to the serial port. The script takes an exclusive flock on /run/das_fanctl.lock and exits rather than interleaving garbage with another writer. Detach attachDAS.sh's screen session before starting the service.
  • DISK_GLOB must name the shelf's HBA on any host with more than one. The default *-sas-exp*-lun-0 means "behind a SAS expander", which is not the same as "in the shelf": a server's own backplane is frequently behind an expander too. On iz-pve0 the internal drives sit on pci-0000:02:00.0 and the shelf on pci-0000:04:00.0, so the default glob matched all 17 drives and would have set the shelf's fans from the server's internal drive temperatures. Set Environment=DISK_GLOB=/dev/disk/by-path/*04:00.0*-lun-0 in the unit.
  • The server's own fan controller needs the mirror-image setting. Whichever host the shelf is plugged into must set DISK_EXCLUDE_PATTERN to that same HBA in fan_speed.service, or the shelf's drives drive the server fans as well. Moving the shelf between hosts means moving both settings.
  • PORT is per host too. Check which UART actually answers rather than assuming: grep -v uart:unknown /proc/tty/driver/serial lists the real ones, then send _temp_rd to each and watch for the BlueDress prompt.

Files

File
das_fanctl.sh The fan controller
fan_speed_das.service systemd unit for it
functions.sh Only calculate_interpolated_fan_speed is used here — the rest is iDRAC/IPMI
attachDAS.sh Drop into an interactive screen session on the shelf console
get_das_temp.sh Superseded by das_fanctl.sh temps — it never read the reply back
log_rotate.txt logrotate snippet for the log directory

Credit

functions.sh comes from the Dell iDRAC fan controller project; this repo only uses its calculate_interpolated_fan_speed. Prior art on the MD1200 console specifically: tonybaltovski/md1200-reduce-fans-systemd and iamjoshgilman/md1200-fan-controller.