Commit Graph

2493 Commits

Author SHA1 Message Date
Aaron Kimbrell
ab395a1af9 feat(net): WORLD_FILES, WORLD_RELOAD and WORLD_FILES_STATUS master messages
Appended at the end of MessageType::Master (47-49); the pin test is
updated. A world sends its zone file list to master once it is ready.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:51:21 -05:00
Aaron Kimbrell
e709855b7d feat(world): record the zone files a world loads, with sizes and hashes
ZoneFileLog keeps the .luz, .lvl, .lutriggers, terrain .raw and navmesh a
world read while loading its zone: absolute path of the loose file (or
the name in the pack), size and an FNV-1a hash of the bytes read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:51:21 -05:00
Aaron Kimbrell
b629766182 fix(mail): new mail notices in live's form
Live filled a NewMail notice with the mail it was about: the mail ID, the
player, the attachment (LOT -1 without one) and a count of 1, and at load sent
one notice per unread mail. DLU sent one notice with every field 0 and the
unread total as the count. The answer to NotificationRequest names the player
and no mail, with LOT -1 and the unread count, as live's did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:51:21 -05:00
Aaron Kimbrell
3f0e9bc336 fix(dashboard): the Network diagram keeps its width on phones and scrolls sideways
Six columns squeezed into a phone's width drew the boxes over each other; the diagram now keeps at least
1100 px and its card scrolls, with the lists and tooltips following the scroll.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:51:21 -05:00
Aaron Kimbrell
230e5b4841 docs(dashboard): Network ports and servers on several machines
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:39:24 -05:00
Aaron Kimbrell
b0819df5b6 feat(dashboard): Network summary has each server's port and machine
The dashboard keeps master's server list endpoints and joins them into the
Network summary: its listening port and its machine as a token, since the
summary goes to every open page. The connection list maps the tokens to
addresses for viewers with network_ips. The dashboard reports its web port.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:39:24 -05:00
Aaron Kimbrell
a93f5f810b feat(master): the server list names each server's port and machine
Master keeps the address and port auth, chat, the dashboard and the UGC
server report when they connect and sends them, with its own and every
world's, after the list's world states. A server's machine is the address
master sees its connection come from; loopback and master's external_ip are
master's machine. Servers can report another port than their RakNet one; the
UGC server reports its web port.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:39:24 -05:00
Aaron Kimbrell
93aa88dd86 feat(dashboard): Network page shows ports and frames servers per machine
Boxes are named with their listening port, instance rows show theirs and
player rows their remote ports (with network_ips). With servers on more than
one machine each machine's boxes sit in a labelled frame with its totals,
foldable into one box and remembered per browser; links between machines are
dash-dotted with a diamond half way. Tooltips, details and the List view name
each server's machine. One machine draws no frames.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:39:24 -05:00
Aaron Kimbrell
a110594c9e feat(dashboard): Network model has ports, machines and cross-machine links
Server boxes and instance rows carry the listening port, player rows their
remote ports when the list has them. With servers on more than one machine
the graph lists the machines with their totals, a zone on several machines
is a box per machine, a folded machine is one box, and the layout stacks
each machine's boxes in a framed band; links between machines are marked.
One machine lays out as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:39:24 -05:00
Aaron Kimbrell
3ea08daa19 fix(mail): tell the mailbox OpenMail and CloseMail like live
Live followed pushGameState with NotifyClientObject OpenMail to the mailbox, and
ToggleMail with CloseMail. The mailbox's client script uses them to mark itself
in use (no interact icon, no second use) while the Mail UI is open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:34:40 -05:00
Aaron Kimbrell
7f96a0a406 test(mail): pin mail packet and mailbox message bytes
The empty and one-entry mail list, the notification layout and the mailbox's
pushGameState/ToggleMail/OpenMail/CloseMail messages, checked against the
client's layout and live captures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:34:40 -05:00
Aaron Kimbrell
04f73a4e5b docs(cdclient): fdb copies and hot reload
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:26:37 -05:00
Aaron Kimbrell
71f1a6076f feat(gm): /reloadcdclient with the cdclient_reload permission (GM 9)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:26:28 -05:00
Aaron Kimbrell
e19f6c5fdc feat(master): watch the client's cdclient.fdb and reload it on every server
Master copies the fdb at startup, polls its size and mtime every
cdclient_watch_seconds, builds the new copy and CDServer.sqlite on a
worker, switches itself, writes the pointer file, tells every world and
keeps the last two copies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:26:28 -05:00
Aaron Kimbrell
1f87b2557d feat(cdclient): servers open master's fdb copy, never the client's file
Worlds, the UGC server and the dashboard open the pair named in
resServer/cdclient-current (CDServer.sqlite without an fdb when there is
none). Worlds switch to a new pair on CDCLIENT_RELOAD between frames.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:20:20 -05:00
Aaron Kimbrell
b05b84ff30 feat(net): CDCLIENT_RELOAD master message
Appended at the end of MessageType::Master (46); the pin test is updated.
With names it tells a server which fdb copy and CDServer.sqlite to switch
to; without them it asks master to check the client's fdb now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:20:20 -05:00
Aaron Kimbrell
27a4564fec feat(cdclient): make a new CDServer.sqlite off-thread and swap tables in place
CDClientSnapshot converts a copy of the client's fdb into its own
CDServer-<hash>.sqlite with the cdserver migrations applied, on its own
connection. CDClientDatabase::Reconnect opens the new file before letting go
of the old one; CDClientManager::Reload empties every table (old entries kept
alive so references from spawned objects stay valid), retires the old fdb view
and loads again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:20:20 -05:00
Aaron Kimbrell
1e0fc92f2e refactor(cdclient): FdbToSqlite can convert into a connection of its own
So a worker can make a new CDServer.sqlite without touching the shared
CDClient connection or the logger.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:20:20 -05:00
Aaron Kimbrell
1a8e4e387c feat(cdclient): content-addressed copies of the client's cdclient.fdb
FdbSnapshot names copies cdclient-<hash>.fdb / CDServer-<hash>.sqlite, writes
them under temporary names and renames them in, keeps a pointer file naming
the current pair, removes old copies, and decides when a changed file has
settled enough to hash. Nothing in it logs, so it can run on a worker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:20:20 -05:00
Aaron Kimbrell
2e7bea3be8 fix(dashboard): a closed Game clients box keeps its links to worlds with idle players
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:15:31 -05:00
Aaron Kimbrell
f86f173cae fix(dashboard): Network page in six columns with chat in its own, boxes that leave room, links spread along each side
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:14:40 -05:00
Aaron Kimbrell
bc45cd444d fix(dashboard): Network boxes are wider and names wrap onto a second line
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:12:35 -05:00
Aaron Kimbrell
79d69bf4ac fix(master): a rocket launch preps the clone the player is going to
Launching to a property started an extra clone 0 instance of the zone besides the property's own, because
the prep only carried the zone. PrepZone now carries the clone when there is one (written only then, so a
plain prep is unchanged).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:11:20 -05:00
Aaron Kimbrell
1147b61fcb docs(readme): Performance page, game text from the locale, capture and replay, TacArc targets, the mapped fdb
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:42:22 -05:00
Aaron Kimbrell
cf03a0c9e9 fix(settings): list the packet capture and replay sandbox settings in the shipped ini files
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:34:35 -05:00
Aaron Kimbrell
b6518caa9c test(capture): fixtures check client game messages, with a synthetic fixture
The fixture check now also reads every recorded client game message with the
struct the server reads it with (GameMessageHandler::CreateReceived) and writes
it again: it must read the whole message and give back the same bits. Game
message fields are decoded with the server's structs in the fixture tests.

A synthetic fixture, built in the test from the server's own structs (login,
position update, a client game message), goes through the export steps
(portable, anonymised, saved, read again) and passes the same checks; recorded
fixtures stay in tests/fixtures-local and are never committed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:34:35 -05:00
Aaron Kimbrell
0b58b3ec97 test: clear the game globals the test fixture deletes
GameDependenciesTest deleted the logger, config and managers but left the
pointers set, so a later test without the fixture (packet capture) logged
through a freed logger and crashed when the whole suite ran.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:34:34 -05:00
Aaron Kimbrell
9043f97622 fix(capture): start and stop the replay sandbox on Windows too
The sandbox launched its servers with fork, setpgid and waitpid only. Starting,
waiting and stopping now go through one small platform section: POSIX keeps
the process group, Windows uses CreateProcess in a job object that is ended as
one (and dies with the tool). Environment overrides are cleared in the tool
itself, program names get .exe on Windows, and shared folders fall back to a
copy where links can't be made.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:34:34 -05:00
Aaron Kimbrell
8680195da6 fix(capture): replay retries the first auth connection
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:34:34 -05:00
Aaron Kimbrell
1f5c63dba2 fix(settings): describe the replay sandbox's settings in the catalog
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:34:34 -05:00
Aaron Kimbrell
31f1bcf2d3 fix(capture): replay keeps its records, skips RakNet's own messages and waits for transfers
- the expected answers pointed into a copy of the records that was gone by the diff
- the client's recorded connect and disconnect messages are RakNet's, not sent again
- a recorded move to another world is waited for as long as a zone takes to start
- the handshake sends the client net version; reports are written after every bundle
- a crash or Ctrl-C of the tool stops the sandbox stack too

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:34:34 -05:00
Aaron Kimbrell
3a8838d463 feat(capture): replay bundles against a sandbox stack with a headless client
CaptureTool (built next to the servers) replays packet bundles and compares the answers:

- replay: per bundle a fresh sandbox folder with copied server binaries, rewritten settings
  (replay_sandbox=1, a new SQLite file inside the folder, ports from --port on, no dashboard),
  read back before anything starts; sandbox-setup runs inside it to apply migrations and make
  the replay account and the bundle's characters (setup mode); master is started, the stack is
  stopped as one process group and the folder deleted unless kept
- with replay_sandbox=1 every server refuses a database that isn't SQLite, isn't inside its
  own folder, or is replay_live_sqlite_path (Database::Connect); replay-target against a
  running server needs --i-know-this-is-not-a-sandbox
- the fake client splits the recording into connections, logs in and picks the character
  itself when the recording doesn't, fills in the target's account, session key and IDs,
  learns server-made object IDs from replica constructions by LOT, follows the recorded
  timing and waits for the answers a client waits for; the diff pairs answers by name (and
  constructions by LOT) and ignores fields that differ between runs
- import-live converts the 2014 live captures (folders of *_traffic.zip; pcaps and encrypted
  captures are left alone) into bundles, with secrets removed and CREATE_CHARACTER as setup
- anonymise makes local fixtures; docs/CaptureReplay.md describes capture, the bundle format,
  portability rules, the sandbox and the replay

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:34:34 -05:00
Aaron Kimbrell
88a97b893e feat(dashboard): play packet captures back and replay their movement in World 3D
The Packet Captures page (/inspector/packets, dev_message_inspector) arms captures with a
search picker (part of an account or character name, or a pasted account or object ID; online
ones marked; everything needs no pick), lists running and saved ones live (packet_capture
topic), and plays one back on its timeline: play, pause, seek and speed, packets appearing in
order with their decoded fields and bytes. World 3D takes ?capture=<id>&zone=<zone> and replays
the captured position updates in its replay mode; the capture page's playhead drives it over a
BroadcastChannel. Saved packet captures show in the Message Inspector's list and open there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:34:34 -05:00
Aaron Kimbrell
332bc04ce8 feat(capture): record whole packets of an account, a character or everything on every server
Staff arm a packet capture on the dashboard; master passes MESSAGE_CAPTURE_CONTROL ARM to
every world, auth and chat and arms its own. Each server's PacketCapture tap (dServer receive,
and a send hook in RakPeer::Send so replica constructions are seen too) records into one
preallocated chunk per server and ships sealed chunks through master on the main loop when
capture_flush_bytes or capture_flush_interval_ms is reached; past capture_buffer_max_mb the
oldest chunks are dropped and the dashboard records a gap. Nothing is armed: one flag check.

- targets: an account (from its login; packets before the login are kept per connection
  and added once auth or the world knows whose they are), a character (from when it is
  picked), or everything; up to 8 at once (a bit each in the record mask)
- worlds and auth record their clients' packets and the master link messages of a captured
  player (session keys by name, zone transfers by request, player added/removed, migration);
  chat finds the player in each packet; master records server traffic for everything
- secrets are never recorded: structs that carry them (login request, login response user
  key, world validation session key, session key messages between servers) are read,
  blanked and written again before recording; auth keeps only the handshake and login
- PacketDecoder: a registry by service and message id names every packet and decodes the
  registered structs; CaptureBundle is the file format (DLUBNDL1, metadata, records);
  CaptureTools orders records on one timeline, pulls movement out, makes bundles portable
  or anonymous and diffs replays
- the dashboard keeps packet captures in message_capture_sessions (capture_kind 1) and
  their packets in a file under capture_dir, one write per batch; arming is audited
- MESSAGE_CAPTURE_CONTROL/DATA only gain appended enum values and trailing fields

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:34:34 -05:00
Aaron Kimbrell
fb6d73e4bd feat(web): API key traffic is its own Network row, named after the key
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
355d997333 docs(dashboard): Network groups open in place; web clients by signed-in user
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
c57ef49b15 feat(dashboard): Network groups open in place as a filterable list in their box
An open group (game clients, web clients, a zone's instances) keeps one box
that grows by its rows, at most 8 in view with the rest scrolled. Each row
in view is a link end of its own with only its member's traffic, and the
open box's header has none. Members are filtered by name, account or user,
instance and (with network_ips) address, in a steady order; web clients
show the dashboard user.

The filter and rows are HTML over the SVG, made once and then moved and
updated in place, so live updates keep the typed text, the focus and the
scroll; links follow the list's scroll. The box grows and shrinks with its
list, and the List view nests the members under their group with the same
filter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
2fb68cd20e feat(dashboard): Network connections name the web client's dashboard user
Web clients are one entry per address and signed-in account, with user and
account_id; the user is shown without network_ips too, the address then
masked as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
569ac6fb31 feat(web): count HTTP clients by the account they were signed in as
Each address's requests are counted per signed-in account (the dashboard's
session or API key, as the auth middleware found it), so several people
behind one address stay apart, with the account's user name; a WebSocket
upgrade counts under its account.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
7c37fca9fb docs(dashboard): Performance page, slow frames, profiling and Tracy
How frame timing is measured and reported, the page, the API and connecting Tracy's viewer. Task 96.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
992abf9d8a feat(dashboard): Performance page with frame times, slow frames and flame graphs
The dashboard's own loop is framed too, with a scope per module update. Frame time and stacked phase charts per server, the servers' loop summary, longest frames, packet handling times, the last 50 slow frames with a nested timeline, and profiling sessions (profiling_run, GM 8) drawn as a flame graph with folded stacks to download. PerfHistory keeps it in memory and is unit tested; the layouts are tested with node. Task 96.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
d4e9423264 feat(servers): time the main loops' phases and the known heavy spots
Frames and phase scopes in the world, auth, chat and UGC loops (packets per type, entities, physics, ghosting, replica, spawners, log flush, saves, web requests by route). Scopes at LoadPlayer, CreateEntity, each component's construction, InventoryComponent::LoadXml, script timers and a world's zone load; game database queries in the query helpers; CDClient statements timed through sqlite3_trace_v2 as CDClient <table> (CppSQLite3DB gets a handle accessor). Task 96.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
63666c3c6b feat(servers): report frame timing, log slow frames, answer profiling requests
dServer sends the frames with each traffic report, logs a frame over slow_frame_ms (default 250, reread on a settings reload) as one line with its heaviest path, and runs profiling sessions master forwards. Master routes the dashboard's requests to the named server, profiles itself and passes results on. Task 96.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
d0c7b089ff feat(net): frame timing section in SERVER_TRAFFIC, profiling messages
An optional section after marker 3 carries the report's frame timing; older readers stop before it and reports without it still read. Phase times go with their count so readers with fewer or more phases read them. PROFILE_REQUEST and PROFILE_RESULT are appended to the master messages (44, 45). Task 96.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
06af6c3aaa feat(profiler): frame timing and scope trees of a server's main loop
Profiler.h: frames, named scopes and phases, recorded on the main thread only (other threads' scopes do nothing). Per second: frames, total and longest frame time, a mergeable frame time histogram (TrafficStats::Histogram) and time per phase. Per report: the packet types that took longest, the worst frames and the frames over a threshold with their heaviest scopes. Profiling sessions merge every frame's scope tree for a while into one (folded stacks for flame graphs). Optional Tracy client (DLU_TRACY, off by default) gets the same frames and scopes. Task 96.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
6758b32605 docs(dashboard): game text and languages
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:47 -05:00
Aaron Kimbrell
6230d35518 test(dashboard): fail when game text is written into the pages
GameTextJs scans templates, scripts and the routes' strings for the
game's zone, item and character names and currency labels, with an
allowlist for legitimate uses and a self check of the scanner.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:47 -05:00
Aaron Kimbrell
597269b4c5 fix(dashboard): coins, universe score, imagination, reputation and life from the locale
Pages label the game's currencies and stats with the client's words
(UI_COINS, UI_UNIVERSE_SCORE, UI_IMAGINATION, UI_REPUTATION, UI_HEALTH)
through game.terms and GameText.term instead of writing them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:47 -05:00
Aaron Kimbrell
ade116c985 fix(dashboard): zone names in help text and pages come from the locale
Settings help names zones by locale key (expanded when served), the
Property Rent page and the Worlds table name zones in the viewer's
language, and the UGC page drops its own inventory name table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:47 -05:00
Aaron Kimbrell
1f8dd73e0b refactor(dashboard): routes name zones, objects and phrases through GameText
The routes' own zone name helpers and the CDClient's English zone and
leaderboard header copies give way to GameText, so every name comes from
the locale in the viewer's language; the Network page's world labels,
mail, missions, leaderboards, item info (cached per language) and the
settings/vanity help strings too. Reward code 4 and the plaque text name
their zone from the locale.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:47 -05:00