Commit Graph

185 Commits

Author SHA1 Message Date
Aaron Kimbrell
d1ea2992e4 feat(dashboard): capture viewer decodes game messages and replica packets
Links the game for its message structs, reads the components of every LOT at startup, and decodes a capture's
replica packets once, in timeline order, on a worker thread.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 06:32:35 -05:00
Aaron Kimbrell
4c93c8fea7 feat(dashboard): draining (old version) badge; properties update when empty
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 04:29:54 -05:00
Aaron Kimbrell
d71e44f3e1 feat(world-reload): keep properties until empty
Busy property instances on changed files are kept (KEEP_UNTIL_EMPTY) instead
of saved, frozen and moved. Every instance acted on is marked outdated; the
files status carries the flag (bit 4).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 04:29:54 -05:00
Aaron Kimbrell
c6831ac80e fix(dashboard): light the UGC model view like the game
Metal was drawn with metalness 1 in the room environment at half strength with one
sun, so metal groups came out nearly black. The game's metal keeps the vertex
color (lit, plus a reflection tinted by it), so metal is now part metal with a
stronger reflection, the environment is brighter and a sky over ground fill
lights the sides away from the sun.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 03:50:37 -05:00
Aaron Kimbrell
826ba058ff feat(dashboard): world change markers on the packet capture timeline
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 03:21:53 -05:00
Aaron Kimbrell
9bbdc45f4f feat(world3d): follow a player across worlds, live and in capture playback
Live: when the position feed reports the followed player only in another world,
load its scene, keep following and toast where they went. Capture playback: load
every zone's movement, switch the scene at the followed character's world change
(forward and when seeking back), follow a lone captured character from the start,
and mark world changes on the replay slider.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 03:21:53 -05:00
Aaron Kimbrell
12c5b34a5d feat(world3d): scene switch and world marker rules for following across worlds
followedMove (live feed), worldAt/captureSwitch (capture playback), worldMarkers and
markersHtml, with node tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 03:21:53 -05:00
Aaron Kimbrell
601a27343f feat(dashboard): capture worlds route, every zone's movement in positions
/api/inspector/sessions/:id/worlds lists the captured characters' moves between
world servers with the zone's name from the locale; /positions also returns them,
takes ?zone=all and names each track's character and zone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 03:21:53 -05:00
Aaron Kimbrell
139df9ab9a feat(dashboard): world files card with changed-on-disk badges and Reload
On the Instances page, behind world_reload; zone names from GameText.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:51:21 -05:00
Aaron Kimbrell
16545d573e feat(master): watch world zone files and replace instances that loaded old ones
Master hashes each reported file on a worker, polls size and mtime every
world_watch_seconds (a change must hold still for one poll), and replaces
stale instances with the instance migration (Mythran shift, or seamless
with world_reload_seamless=1); empty ones are stopped. Also on WORLD_RELOAD.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:51:21 -05:00
Aaron Kimbrell
3f0e9bc336 fix(dashboard): the Network diagram keeps its width on phones and scrolls sideways
Six columns squeezed into a phone's width drew the boxes over each other; the diagram now keeps at least
1100 px and its card scrolls, with the lists and tooltips following the scroll.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:51:21 -05:00
Aaron Kimbrell
b0819df5b6 feat(dashboard): Network summary has each server's port and machine
The dashboard keeps master's server list endpoints and joins them into the
Network summary: its listening port and its machine as a token, since the
summary goes to every open page. The connection list maps the tokens to
addresses for viewers with network_ips. The dashboard reports its web port.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:39:24 -05:00
Aaron Kimbrell
93aa88dd86 feat(dashboard): Network page shows ports and frames servers per machine
Boxes are named with their listening port, instance rows show theirs and
player rows their remote ports (with network_ips). With servers on more than
one machine each machine's boxes sit in a labelled frame with its totals,
foldable into one box and remembered per browser; links between machines are
dash-dotted with a diamond half way. Tooltips, details and the List view name
each server's machine. One machine draws no frames.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:39:24 -05:00
Aaron Kimbrell
a110594c9e feat(dashboard): Network model has ports, machines and cross-machine links
Server boxes and instance rows carry the listening port, player rows their
remote ports when the list has them. With servers on more than one machine
the graph lists the machines with their totals, a zone on several machines
is a box per machine, a folded machine is one box, and the layout stacks
each machine's boxes in a framed band; links between machines are marked.
One machine lays out as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:39:24 -05:00
Aaron Kimbrell
e19f6c5fdc feat(master): watch the client's cdclient.fdb and reload it on every server
Master copies the fdb at startup, polls its size and mtime every
cdclient_watch_seconds, builds the new copy and CDServer.sqlite on a
worker, switches itself, writes the pointer file, tells every world and
keeps the last two copies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:26:28 -05:00
Aaron Kimbrell
1f87b2557d feat(cdclient): servers open master's fdb copy, never the client's file
Worlds, the UGC server and the dashboard open the pair named in
resServer/cdclient-current (CDServer.sqlite without an fdb when there is
none). Worlds switch to a new pair on CDCLIENT_RELOAD between frames.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:20:20 -05:00
Aaron Kimbrell
2e7bea3be8 fix(dashboard): a closed Game clients box keeps its links to worlds with idle players
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:15:31 -05:00
Aaron Kimbrell
f86f173cae fix(dashboard): Network page in six columns with chat in its own, boxes that leave room, links spread along each side
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:14:40 -05:00
Aaron Kimbrell
bc45cd444d fix(dashboard): Network boxes are wider and names wrap onto a second line
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:12:35 -05:00
Aaron Kimbrell
1f5c63dba2 fix(settings): describe the replay sandbox's settings in the catalog
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:34:34 -05:00
Aaron Kimbrell
88a97b893e feat(dashboard): play packet captures back and replay their movement in World 3D
The Packet Captures page (/inspector/packets, dev_message_inspector) arms captures with a
search picker (part of an account or character name, or a pasted account or object ID; online
ones marked; everything needs no pick), lists running and saved ones live (packet_capture
topic), and plays one back on its timeline: play, pause, seek and speed, packets appearing in
order with their decoded fields and bytes. World 3D takes ?capture=<id>&zone=<zone> and replays
the captured position updates in its replay mode; the capture page's playhead drives it over a
BroadcastChannel. Saved packet captures show in the Message Inspector's list and open there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:34:34 -05:00
Aaron Kimbrell
332bc04ce8 feat(capture): record whole packets of an account, a character or everything on every server
Staff arm a packet capture on the dashboard; master passes MESSAGE_CAPTURE_CONTROL ARM to
every world, auth and chat and arms its own. Each server's PacketCapture tap (dServer receive,
and a send hook in RakPeer::Send so replica constructions are seen too) records into one
preallocated chunk per server and ships sealed chunks through master on the main loop when
capture_flush_bytes or capture_flush_interval_ms is reached; past capture_buffer_max_mb the
oldest chunks are dropped and the dashboard records a gap. Nothing is armed: one flag check.

- targets: an account (from its login; packets before the login are kept per connection
  and added once auth or the world knows whose they are), a character (from when it is
  picked), or everything; up to 8 at once (a bit each in the record mask)
- worlds and auth record their clients' packets and the master link messages of a captured
  player (session keys by name, zone transfers by request, player added/removed, migration);
  chat finds the player in each packet; master records server traffic for everything
- secrets are never recorded: structs that carry them (login request, login response user
  key, world validation session key, session key messages between servers) are read,
  blanked and written again before recording; auth keeps only the handshake and login
- PacketDecoder: a registry by service and message id names every packet and decodes the
  registered structs; CaptureBundle is the file format (DLUBNDL1, metadata, records);
  CaptureTools orders records on one timeline, pulls movement out, makes bundles portable
  or anonymous and diffs replays
- the dashboard keeps packet captures in message_capture_sessions (capture_kind 1) and
  their packets in a file under capture_dir, one write per batch; arming is audited
- MESSAGE_CAPTURE_CONTROL/DATA only gain appended enum values and trailing fields

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:34:34 -05:00
Aaron Kimbrell
c57ef49b15 feat(dashboard): Network groups open in place as a filterable list in their box
An open group (game clients, web clients, a zone's instances) keeps one box
that grows by its rows, at most 8 in view with the rest scrolled. Each row
in view is a link end of its own with only its member's traffic, and the
open box's header has none. Members are filtered by name, account or user,
instance and (with network_ips) address, in a steady order; web clients
show the dashboard user.

The filter and rows are HTML over the SVG, made once and then moved and
updated in place, so live updates keep the typed text, the focus and the
scroll; links follow the list's scroll. The box grows and shrinks with its
list, and the List view nests the members under their group with the same
filter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
2fb68cd20e feat(dashboard): Network connections name the web client's dashboard user
Web clients are one entry per address and signed-in account, with user and
account_id; the user is shown without network_ips too, the address then
masked as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
992abf9d8a feat(dashboard): Performance page with frame times, slow frames and flame graphs
The dashboard's own loop is framed too, with a scope per module update. Frame time and stacked phase charts per server, the servers' loop summary, longest frames, packet handling times, the last 50 slow frames with a nested timeline, and profiling sessions (profiling_run, GM 8) drawn as a flame graph with folded stacks to download. PerfHistory keeps it in memory and is unit tested; the layouts are tested with node. Task 96.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
63666c3c6b feat(servers): report frame timing, log slow frames, answer profiling requests
dServer sends the frames with each traffic report, logs a frame over slow_frame_ms (default 250, reread on a settings reload) as one line with its heaviest path, and runs profiling sessions master forwards. Master routes the dashboard's requests to the named server, profiles itself and passes results on. Task 96.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
597269b4c5 fix(dashboard): coins, universe score, imagination, reputation and life from the locale
Pages label the game's currencies and stats with the client's words
(UI_COINS, UI_UNIVERSE_SCORE, UI_IMAGINATION, UI_REPUTATION, UI_HEALTH)
through game.terms and GameText.term instead of writing them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:47 -05:00
Aaron Kimbrell
ade116c985 fix(dashboard): zone names in help text and pages come from the locale
Settings help names zones by locale key (expanded when served), the
Property Rent page and the Worlds table name zones in the viewer's
language, and the UGC page drops its own inventory name table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:47 -05:00
Aaron Kimbrell
1f8dd73e0b refactor(dashboard): routes name zones, objects and phrases through GameText
The routes' own zone name helpers and the CDClient's English zone and
leaderboard header copies give way to GameText, so every name comes from
the locale in the viewer's language; the Network page's world labels,
mail, missions, leaderboards, item info (cached per language) and the
settings/vanity help strings too. Reward code 4 and the plaque text name
their zone from the locale.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:47 -05:00
Aaron Kimbrell
17b9479c0e feat(dashboard): every request and page gets game text in the viewer's language
The dashboard loads every language in the client's locale; each request
runs in the viewer's language (Workers::Reply carries it to the worker),
templates get game.* and phrase()/zone_name(), scripts get GameText.* on
<body data-game>, and the user menu picks the game text language.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:46 -05:00
Aaron Kimbrell
a6fc67a4bb feat(dashboard): GameText, the game's text in the viewer's language
One helper for game text on the dashboard: zone names, locale phrases,
localized table columns, %[key] expansion and the game's currency words,
looked up in the viewer's language (a cookie pick, else Accept-Language),
falling back to en_US and then the key or id. Unit tested.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:46 -05:00
Aaron Kimbrell
381800fe5e feat(dashboard): Network groups open and close smoothly
A labelled button (▸ 3 instances, ▸ each, ▾ group) or a double click opens a group; its members stack
where the group is (or was dragged to), boxes glide to their new places and new ones fade in. Rows keep
a fixed pitch, so opening a group only moves its own column, and open groups are remembered per browser.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:30:31 -05:00
Aaron Kimbrell
c44af311ad feat(dashboard): client system info, as reported by the client
Account pages get a Client system info (as reported) card (client_sysinfo):
every description the account's client sent at login, newest first, with the
raw values, the memory text split into numbers and each field's caveat. The
address is only shown with logs_audit. The Client System Info page (Logs &
Health) shows the spread across players from each account's newest report:
Windows version, video card, memory buckets, processor count and client build,
marked as client-reported and approximate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:09:22 -05:00
Aaron Kimbrell
05a99cef82 feat(dashboard): client_sysinfo permission and retention
client_sysinfo (GM 5, grantable) shows the client system info; the Log pruning
task deletes rows not seen for log_client_sysinfo_days (90).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:09:14 -05:00
Aaron Kimbrell
55775d9623 feat(auth): keep the client's system info at login
After a successful login the auth server stores the system description the
client sent (memory text, video card, processor fields, Windows version fields)
exactly as sent, on the main thread with the login's other writes. The address
is only kept while log_login_addresses is on; log_client_sysinfo (on) turns the
whole thing off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:09:14 -05:00
Aaron Kimbrell
1e5660512e refactor(chat): remove the chat server's web API
The dashboard's API covers what it offered (online players, teams, announcements) with API keys, so the
chat server no longer runs a web server; its settings show as unused on the settings page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:09:02 -05:00
Aaron Kimbrell
8948e2e81a fix(dashboard): Network links follow dragged boxes; Diagram or List, no Auto
Links leave and enter on the sides that face each other (top and bottom when one box is above the
other), so moving a box no longer leaves links ending in the air.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:07:27 -05:00
Aaron Kimbrell
9b00e437e8 fix(dashboard): Network connections are one entry per player or per connection, not per address
A player on the servers' machine (or behind the same address as other players) was merged with the
servers' own links and everyone else there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:07:27 -05:00
Aaron Kimbrell
a824ff9914 feat(dashboard): drag the Network page's boxes around; links coloured against the busiest link
The layout the viewer makes is kept in the browser, with a Reset layout button. A link's colour now
compares it with the busiest link right now instead of its own recent peak, which made steady links
look saturated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:41:35 -05:00
Aaron Kimbrell
6a59da1748 fix(dashboard): web clients sit right of the dashboard on the Network page
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:38:17 -05:00
Aaron Kimbrell
e6b7677c1f feat(dashboard): Network page with a live traffic flow diagram
Next to Diagnostics (health_view): game and web clients, auth, chat,
worlds per zone (expandable), other reporting servers, master, the
dashboard and the UGC server, with a lane each way per link sized by
bytes/s, animated by packets/s and coloured by load against its own
5 minute peak. Estimated links (servers without a split) are dashed.
Clicking a box shows its links, busiest message types and a sparkline,
with a link to Diagnostics (which now takes ?server=). A connections
table lists each remote address; single clients can be drawn too.
Narrow screens get a list. Drawing stops while the tab is hidden.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:28:39 -05:00
Aaron Kimbrell
7e87f2a482 feat(dashboard): network summary, server detail and connection routes
The traffic topic now carries each server's rates with its split by
peer (null for servers that report none), link statistics and gauges.
New routes: /api/diagnostics/network, /network/server (message types
and a 10 minute series) and /network/connections (remote ends grouped
by address). Addresses need the new network_ips permission; without it
each is a salted token. They stay in memory from the last report only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:28:39 -05:00
Aaron Kimbrell
9e8f161dd8 feat(traffic): count traffic by peer and by connection
Every server now splits its packet counts by peer: its own connections
(players on auth and worlds), its master link, or other servers (the
worlds on chat, every server on master, a world's chat link, which is
now counted too). HTTP requests carrying X-Darkflame-Server count as
another server's, the dashboard counts its own requests to the UGC
server, and each HTTP client address is counted. The report also gets
each RakNet connection's statistics (worlds name the player on it),
trimmed to the 32 busiest with the rest summed. Counting stays on the
main loop, except the dashboard's UGC fetches, which only touch the
locked recorder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:28:39 -05:00
Aaron Kimbrell
6c43753fb2 feat(dashboard): chat flags queue
Staff flag chat for review (chat_flag): one message or several from one
conversation, with who it is about and a note. The flag keeps a copy of the
messages and 10 before and after. The Chat Flags page (Inbox, badge for open
flags) filters by status, character and account; a flag links to the character,
the account (mute, warn, ban), their player reports, their other flags and the
conversation, and shows its history. Reviewers (chat_flag_review) mark it
actioned or dismissed, change the note, link a player report or comment; all of
it is in the flag's history and the audit log. The sidebar adds Team Chat.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:23:23 -05:00
Aaron Kimbrell
f3a49991d4 feat(dashboard): guild, team and whisper chat histories
Guild chat per guild (/chat_log/guild/<id>, from the guild's card), team chat by
team (/chat_log/teams) and a character's whispers by conversation
(/characters/<id>/whispers, chat_dms; opening one is audited), paged on the
server. The Chat Log filters by account and time range, shows the conversation
around a message, links to its history and marks flagged messages; messages can
be picked (shift-click for a range) to flag. The character page links to all of
them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:23:23 -05:00
Aaron Kimbrell
6fd0dbd723 feat(dashboard): keep chat for 30 days by default
log_chat_days defaults to 30 (was 90). Pruning stays on the Log pruning task's
own database connection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:23:23 -05:00
Aaron Kimbrell
8e440d4928 feat(dashboard): reading whispers is its own permission
chat_dms (GM 8) reads whispers; chat_private is now team and guild chat only.
Adds chat_flag and chat_flag_review (GM 3). Both can be granted per account or
character with the permission grants.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:23:22 -05:00
Aaron Kimbrell
063361711b feat(dashboard): pick and compare the processor
Make again gets a Made by picker, /api/ugc/options lists the processor
choices and default, the settings page has the toolbox_* settings, and
/reprocessproperty takes native or toolbox-blender.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 16:47:40 -05:00
Aaron Kimbrell
744bee40df feat: log the build identifier at startup and show it on the dashboard About page
Every server logs BuildInfo's build string as its version; the About page
also shows the build kind and commit, and log bundles name the build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 16:29:46 -05:00
Aaron Kimbrell
db63fd2919 feat(ugc): ray_backend=embree-gpu traces on Intel GPUs with Embree's SYCL (optional build)
The third ray backend, so every machine has a library for it: Embree on x86
CPUs (embree), HIPRT on AMD and NVIDIA GPUs (hiprt), Embree through SYCL on
Intel Arc and Xe GPUs (embree-gpu).

- CMake option DLU_EMBREE_SYCL (off). dUgcServer/EmbreeSycl is a project of
  its own built by a SYCL compiler (DLU_SYCL_CXX; icpx through ONEAPI_ROOT or
  the path, or the open source DPC++'s clang++ through DPCPP_ROOT) as an
  external project: Embree 4.4 with EMBREE_SYCL_SUPPORT, linked statically and
  bound inside (-Bsymbolic, only its C functions exported, so it never meets
  the servers' own Embree), and the GPU kernels (nearest hit skipping a ray's
  triangle, any hit), into libdlu_embree_sycl next to the servers
- UgcRaysEmbreeGpu loads it the first time embree-gpu is asked for; one GPU for
  the process (embree_gpu_device picks it), the workers take turns, the
  occlusion rays in batches as for hiprt
- without the build, the library or a supported Intel GPU it falls back to
  embree and says why (the UGC server's log at start, --make-model on stderr)
- the option names, the settings page, the dashboard's picker, /reprocessproperty

Verified here: the default build and ctest; the SYCL build with the open source
DPC++ 7.1.0 (compiles, links against oneAPI's libsycl.so.9, exports only its C
functions); on this machine (no Intel GPU) it loads, finds no GPU and falls
back to embree. Not verified: tracing on an Intel GPU (none here).

Check: on a machine with an Intel Arc or Xe GPU and oneAPI, configure with
-DDLU_EMBREE_SYCL=ON and run UgcServer --make-model x.lxfml out embree-gpu;
the UGC tests then compare it with Embree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:29:08 -05:00