feat: live updates move every server onto a new build without a restart

With new binaries in place, master moves everything onto them while the
server keeps running (the dashboard's Live update, /liveupdate or SIGUSR2 to
master):

- database migrations of the new build first; a failure stops there
- UGC finishes the jobs it is running (queued rows stay pending), auth
  restarts, chat hands its teams to master for the next chat server; master
  starts the new processes and retries ones that don't come back
- once the new chat server is up (CHAT_SERVER_READY) every world connects at
  once and sends its players again (LoginSessionNotify resync, no login logged)
- every world instance is replaced with an instance migration: public worlds
  and private ones (same password) at once, properties after the old instance
  saved and froze the property (MIGRATE_PREPARE: no building, claiming or
  saving there any more), activity zones and character select once their
  players left or after a wait; empty instances just stop, zones in
  prestart_worlds get a new one first
- the dashboard restarts last and picks the status up again

Players land where they stood (position carried in CarriedPlayerState, also on
properties and Moon Base). Draining instances get no new players
(InstanceMigration::AcceptsNewPlayers) and show as "Moving players" in the
world list. The order lives in LiveUpdateMachine.h without master state and is
unit tested; master's glue is LiveUpdateCoordinator. Master itself is not
replaced. Message IDs are appended only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Aaron Kimbrell
2026-09-28 15:01:05 -05:00
parent bfc45dc662
commit 109a936798
47 changed files with 3213 additions and 59 deletions

View File

@@ -22,3 +22,24 @@ enable_dashboard=0
# Start the UGC server, which makes and serves the meshes and icons of what players build (0 = disabled, 1 = enabled)
# Its settings are in ugcconfig.ini; see docs/UgcServer.md for the client's boot.cfg settings
enable_ugc_server=0
# Live updates (docs/LiveUpdate.md): moving every server and world instance onto a new build without a restart,
# started from the dashboard, with /liveupdate or with SIGUSR2 to master. Read when one starts.
# Seconds players are warned before they are moved (0-300)
live_update_warn_seconds=10
# World instances replaced at the same time
live_update_parallel_worlds=4
# Seconds dead or building players may take before they are moved anyway
live_update_player_wait=30
# Seconds builders on a property may take before it is saved for its new instance anyway
live_update_property_build_wait=60
# Seconds players at character select get before they are moved to the new one
live_update_char_select_wait=60
# Seconds races, minigames and other activity zones get to finish before their players are moved
live_update_activity_wait=1800
# Seconds the UGC server may take to finish the models it is making before it is stopped
live_update_ugc_drain_timeout=300
# Seconds auth, chat, the UGC server or the dashboard may take to stop or come back before master starts it again
live_update_service_timeout=30
# Run the new build's database migrations first (0 or 1)
live_update_run_migrations=1