Commit Graph

23 Commits

Author SHA1 Message Date
Aaron Kimbrell
332bc04ce8 feat(capture): record whole packets of an account, a character or everything on every server
Staff arm a packet capture on the dashboard; master passes MESSAGE_CAPTURE_CONTROL ARM to
every world, auth and chat and arms its own. Each server's PacketCapture tap (dServer receive,
and a send hook in RakPeer::Send so replica constructions are seen too) records into one
preallocated chunk per server and ships sealed chunks through master on the main loop when
capture_flush_bytes or capture_flush_interval_ms is reached; past capture_buffer_max_mb the
oldest chunks are dropped and the dashboard records a gap. Nothing is armed: one flag check.

- targets: an account (from its login; packets before the login are kept per connection
  and added once auth or the world knows whose they are), a character (from when it is
  picked), or everything; up to 8 at once (a bit each in the record mask)
- worlds and auth record their clients' packets and the master link messages of a captured
  player (session keys by name, zone transfers by request, player added/removed, migration);
  chat finds the player in each packet; master records server traffic for everything
- secrets are never recorded: structs that carry them (login request, login response user
  key, world validation session key, session key messages between servers) are read,
  blanked and written again before recording; auth keeps only the handshake and login
- PacketDecoder: a registry by service and message id names every packet and decodes the
  registered structs; CaptureBundle is the file format (DLUBNDL1, metadata, records);
  CaptureTools orders records on one timeline, pulls movement out, makes bundles portable
  or anonymous and diffs replays
- the dashboard keeps packet captures in message_capture_sessions (capture_kind 1) and
  their packets in a file under capture_dir, one write per batch; arming is audited
- MESSAGE_CAPTURE_CONTROL/DATA only gain appended enum values and trailing fields

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:34:34 -05:00
Aaron Kimbrell
992abf9d8a feat(dashboard): Performance page with frame times, slow frames and flame graphs
The dashboard's own loop is framed too, with a scope per module update. Frame time and stacked phase charts per server, the servers' loop summary, longest frames, packet handling times, the last 50 slow frames with a nested timeline, and profiling sessions (profiling_run, GM 8) drawn as a flame graph with folded stacks to download. PerfHistory keeps it in memory and is unit tested; the layouts are tested with node. Task 96.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
a6fc67a4bb feat(dashboard): GameText, the game's text in the viewer's language
One helper for game text on the dashboard: zone names, locale phrases,
localized table columns, %[key] expansion and the game's currency words,
looked up in the viewer's language (a cookie pick, else Accept-Language),
falling back to en_US and then the key or id. Unit tested.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:46 -05:00
Aaron Kimbrell
c44af311ad feat(dashboard): client system info, as reported by the client
Account pages get a Client system info (as reported) card (client_sysinfo):
every description the account's client sent at login, newest first, with the
raw values, the memory text split into numbers and each field's caveat. The
address is only shown with logs_audit. The Client System Info page (Logs &
Health) shows the spread across players from each account's newest report:
Windows version, video card, memory buckets, processor count and client build,
marked as client-reported and approximate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:09:22 -05:00
Aaron Kimbrell
7e87f2a482 feat(dashboard): network summary, server detail and connection routes
The traffic topic now carries each server's rates with its split by
peer (null for servers that report none), link statistics and gauges.
New routes: /api/diagnostics/network, /network/server (message types
and a 10 minute series) and /network/connections (remote ends grouped
by address). Addresses need the new network_ips permission; without it
each is a salted token. They stay in memory from the last report only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:28:39 -05:00
Aaron Kimbrell
6c43753fb2 feat(dashboard): chat flags queue
Staff flag chat for review (chat_flag): one message or several from one
conversation, with who it is about and a note. The flag keeps a copy of the
messages and 10 before and after. The Chat Flags page (Inbox, badge for open
flags) filters by status, character and account; a flag links to the character,
the account (mute, warn, ban), their player reports, their other flags and the
conversation, and shows its history. Reviewers (chat_flag_review) mark it
actioned or dismissed, change the note, link a player report or comment; all of
it is in the flag's history and the audit log. The sidebar adds Team Chat.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:23:23 -05:00
Aaron Kimbrell
f3a49991d4 feat(dashboard): guild, team and whisper chat histories
Guild chat per guild (/chat_log/guild/<id>, from the guild's card), team chat by
team (/chat_log/teams) and a character's whispers by conversation
(/characters/<id>/whispers, chat_dms; opening one is audited), paged on the
server. The Chat Log filters by account and time range, shows the conversation
around a message, links to its history and marks flagged messages; messages can
be picked (shift-click for a range) to flag. The character page links to all of
them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:23:23 -05:00
Aaron Kimbrell
431eddd5bd feat(dashboard): guilds page, guild name review and guild chat in the chat log
A Guilds page (Moderation, permission guilds_manage, GM 5 by default): every guild with its name status, member count,
leader and age, a pending-only switch, and per guild its members (with ranks) and history (guild_events). Staff can
approve a guild name that waits for review, reject it (the guild becomes "Guild <id>"), rename a guild (same name rules
as the game, unique without regard to case), remove a member (a leader's guild goes to the next member, the last
member's guild is deleted) and disband a guild. Every change is audited and added to the guild's history, and the chat
server is asked (GUILD_CHANGED through master) to tell the online members. Pending guild names also show in the Review
Queue. Guild chat ("guild" in the chat log) counts as private chat like whispers and team chat (chat_private), with its
own channel filter and Prometheus counter.

Check on the dashboard: /guilds lists a guild made in game; approve/reject/rename/remove/disband update the in-game guild
window and name billboard of online members; the Review Queue shows a guild whose name waits for review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 06:48:40 -05:00
Aaron Kimbrell
821b7c8767 feat(dashboard): permission grants count in every dashboard permission check
What someone may do on the dashboard is now their GM level's permissions plus the grants on their account, minus its
denies (PermissionGrants.h). A deny beats a grant; denies never apply to GM 9, and settings and permissions_manage stay
GM 9 only. The account's grants are read with every request (like its GM level), so a change applies at once, and
they are passed through every check: RouteUtils::Can, CanViewCharacter, the rank rules (self_* and manage_equal_rank),
routes guarded by a permission, the templates' `can`, the API documentation, API access, API key scopes (a key never
does more than its owner may now) and WebSocket subscriptions.

New permission grants_manage (GM 9 by default) and the API to manage grants: GET /api/grants/catalog, GET /api/grants,
POST /api/grants, POST /api/grants/:id/remove. Nobody grants or takes away what they don't hold themselves (a
permission, every permission of a group, a command they may use, every command up to their own GM level), and only on
accounts the rank rules let them manage (their own with self_moderation). Commands with a fixed level or a floor
above GM 1 (/execute) can't be granted. Every change goes in the audit log (grant_permission, deny_permission,
remove_grant). Also: the Showcase gate and the traffic subscription now check their permission by name.

Check: grant a GM 2 account accounts_ban (it can ban, and the Ban button shows); deny a GM 8 account accounts_view (the
accounts list is refused); give an expiry a minute ahead and see it stop; try to grant a permission your account
doesn't have (refused); dWebTests PermissionGrantsTests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 01:16:38 -05:00
Aaron Kimbrell
109a936798 feat: live updates move every server onto a new build without a restart
With new binaries in place, master moves everything onto them while the
server keeps running (the dashboard's Live update, /liveupdate or SIGUSR2 to
master):

- database migrations of the new build first; a failure stops there
- UGC finishes the jobs it is running (queued rows stay pending), auth
  restarts, chat hands its teams to master for the next chat server; master
  starts the new processes and retries ones that don't come back
- once the new chat server is up (CHAT_SERVER_READY) every world connects at
  once and sends its players again (LoginSessionNotify resync, no login logged)
- every world instance is replaced with an instance migration: public worlds
  and private ones (same password) at once, properties after the old instance
  saved and froze the property (MIGRATE_PREPARE: no building, claiming or
  saving there any more), activity zones and character select once their
  players left or after a wait; empty instances just stop, zones in
  prestart_worlds get a new one first
- the dashboard restarts last and picks the status up again

Players land where they stood (position carried in CarriedPlayerState, also on
properties and Moon Base). Draining instances get no new players
(InstanceMigration::AcceptsNewPlayers) and show as "Moving players" in the
world list. The order lives in LiveUpdateMachine.h without master state and is
unit tested; master's glue is LiveUpdateCoordinator. Master itself is not
replaced. Message IDs are appended only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:23 -05:00
Aaron Kimbrell
f65307992e refactor(dashboard): one helper for fetching from the UGC server
The /ugc page's routes and the UGC links' icon and mesh routes each had
their own copy of the curl request and its short cache. Both use
UgcFetch now: the URL is worked out on the web thread, the request and
cache run on worker threads. Players' errors still don't name the
internal URL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:11 -05:00
Aaron Kimbrell
1a2758feab feat(ugc): icon light matched to the game, debounce, shared car icons, purge, icon editor
Brighter icons: a world light, a fill from the camera, a highlight, exposure
and contrast; with the defaults the icons' mean luminance matches the game's
own model icons (118 against 120 on a scratch set). Every icon parameter is
listed once (UgcIconParams: key, setting, range, default); the settings,
the dashboard's settings entries and the icon editor come from that list.
Presets per kind (player models, each car or rocket build type from
ModularBuildComponent) and overrides per model or module combination are in
ugc_icon_settings.

Saved models wait ugc_debounce_seconds (sharedconfig) after the owner's last
save before they're made (ugc.process_after); a client asking for one, the
owner leaving the world or a reset ends the wait.

Cars and rockets: one icon per combination of modules (sorted LOTs), shared
by every build of it; builds of a combination made already are marked made
right away, the client's per-blueprint downloads serve the shared files.

The dashboard can delete one item's files, purge by filter or all, preview
icons with any values on the UGC server (/admin routes, master password),
save presets and overrides, and draw a kind's icons again (icons only).

Migrations dlu/mysql/86 and dlu/sqlite/69. Not done yet: the dashboard
editor's lighting controls in the page script (routes are there), docs for
it, the empty-model state, /ugc?item= links, the shared fetch helper; the
storage size and property loading bugs are next.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:09 -05:00
Aaron Kimbrell
dce1c78336 feat(dashboard): UGC search, and UGC icons on property and character pages
UGC Search (/ugc_search) finds creations by name, id or LOT with where each
one is. Property pages and character inventories show the icon the UGC server
made of each creation, its state and a link to /ugc?item=&kind=, for whoever
may view the page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:08 -05:00
Aaron Kimbrell
709cfab140 feat(dashboard): log bundles, zipped log files picked by date, server and world
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:07 -05:00
Aaron Kimbrell
3f2a9cc5b0 feat(dashboard): scoped API keys with rate limits and quotas
Bearer keys (dlk_...) are checked per request against their owner's
current account (ban, lock, demotion, sign out everywhere stop or narrow
them at once) and their scope: permission routes need the permission in
scope, read-only keys only read, level-only routes need an all-permission
key, and session-only paths (sign-in, password, 2FA, key management)
are never reachable with a key. Per-key rate limit and daily quota with
429 and X-RateLimit/X-Quota/Retry-After headers; usage is written in
batches every minute. WebSocket subscriptions honour the scope too.

Routes to list, make, rotate and revoke keys; staff with
api_keys_manage can see and revoke others' keys under the rank rules.
POST /api/auth/token now makes an all-permission key. Audit entries for
create/rotate/revoke/denied, and actions done with a key are attributed
to "user (key name)".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:03 -05:00
Aaron Kimbrell
e6148a4021 feat(dashboard): Diagnostics page with live traffic of every server
Keeps every server's traffic reports (last hour at one second, per-minute
rows to server_traffic once a minute, pruned after traffic_days) and shows
packets, bytes, HTTP requests and latency per second, per server, with the
busiest message types and HTTP routes. Live over the traffic WebSocket topic;
1 hour, 24 hours and 7 days ranges. The same counters are in /metrics.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:02 -05:00
Aaron Kimbrell
347a0a7286 feat(dashboard): UGC server page
Counts, a paged list and failures of what the UGC server made of players'
models and modular builds, making one, the failed ones or everything again
(new ugc_manage permission), the UGC server's live status and icons from
ugc_public_url, and a 3D view of a model's LXFML. The UGC settings are in the
settings catalog.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:57 -05:00
Aaron Kimbrell
abf1cc1d80 feat: UGC server that makes and serves player models' meshes and icons
A new server (dUgcServer, started by master with enable_ugc_server=1) that
takes unprocessed ugc and ugc_modular_build rows from the database and makes
what the client downloads with UGCUSE3DSERVICES: an optimized NIF (hidden
faces removed, ambient occlusion baked into vertex colors) and a 128px DDS
icon for player models, rendered by a software rasterizer from the client's
LDD brick primitives, and icons for cars and rockets assembled from their
modules per ModularBuildComponent/ModuleComponent. It serves them, with the
models' LXFML, over HTTP in the client's UGCC<dc>/3DOPTIMIZED and
IMAGE128DDS layout with .gz and .checksum files, and keeps its folder under
a size cap.

Processing state lives in ugc.is_optimized plus new processed_at,
process_attempts and process_error columns (and the same on
ugc_modular_build). ServiceType::UGC is appended. NifFile moves to dCommon
and records named node transforms for the modules' attach points.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:57 -05:00
Aaron Kimbrell
680615ba9f feat: optional property rent
Off unless property_rent_enabled is on. Each property world's rent comes from its PropertyTemplate
row (minimumPrice every rentDuration x durationType; Block Yard is free), unless the new Property
Rent dashboard page sets another price or period (property_rent_manage). Rent is taken from the
owner's coins shortly after their character loads, with a mail receipt; unpaid rent is mailed and,
after property_rent_grace_days, makes the property private until it is paid, like live. The
property management component refuses public or best friends privacy while rent is overdue and a
property world that loads overdue makes itself private. Property game messages are unchanged.

Fixes #943

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:52 -05:00
Aaron Kimbrell
b2403b09ea feat: contraband list with flagging and optional removal
Staff list contraband items on a new dashboard page (item search, reason, flag or flag and remove;
contraband_manage to edit, reports_view to see). World servers check every inventory when a
character loads and every item a player receives: each find is an economy flag of the new kind
Contraband, shown with the other flags and in the character's related data. Items marked for
removal are taken away, with a character snapshot kept first so they can be given back, an audit
entry and a mail or chat message telling the player why. Staff are skipped unless
contraband_ignore_staff is off. Worlds reload the list when it changes (RELOAD_CONTRABAND, added
at the end of ePlayerAction).

Fixes #1563

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:52 -05:00
Aaron Kimbrell
17689cd663 feat: build a zone's 3D data on worker threads
Opening a zone the first time built its terrain, scene objects and manifests
and ran ImageMagick on the web thread, stalling the dashboard for seconds.

- Workers: the shared pool plus Workers::Reply (answer at once when built,
  else from a worker via Web::Defer).
- terrain_chunks/terrain_layers/scene/paths/scenery/flairs (world3d, property
  and showcase routes) and terrain textures go through it; results are built
  once in OnceCaches, the .raw is read once per zone for chunks, layers and
  flairs, deflated bodies are cached thread-safely.
- ImageMagick conversions are deduplicated and written under a temporary name.
- Workers don't query the CDClient, read settings or call mongoose: ZoneTable,
  render components, flairs, object names, LOT kinds and terrain texture names
  are read at startup; client_location is read once; base64 is plain C++.
- Logger writes one line at a time (mutex; localtime's buffer is shared).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:51 -05:00
Aaron Kimbrell
23e803280f feat: convert 3D scenery models on worker threads with deferred replies
The dashboard's web server answers one request at a time, so converting a big
.nif (glom files up to tens of MB) held up every other request, flairs included.

- dWeb: Web::Defer hands a request to another thread; the reply is sent from the
  web thread on its next poll (DeferredQueue). A client that leaves first cancels
  it and the late reply is dropped. The synchronous route API is unchanged.
- Web::Shutdown closes connections while the state their close events touch is
  still alive; the destructor no longer runs handlers during static destruction
  (stopping the dashboard aborted in ~WSClient).
- WorkerPool: priority lanes, with one thread only for urgent work (flairs,
  small models, textures), and limited background work.
- Scenery: mesh and texture routes (and the showcase's) convert on the pool;
  thread-safe memory and disk caches, one conversion per model at a time with
  waiters sharing it; zones are converted ahead onto the disk cache while viewed.
- Setting scenery_workers (0: half the cores, 2 to 4).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:48 -05:00
Aaron Kimbrell
e213a7aeff feat: web dashboard and playground work
The NexusDashboard-parity dashboard (dDashboardServer) and everything built on it on the experimental branch:
accounts, characters, properties and moderation tools, permissions shared with in-game slash commands, economy
reports, World 3D and property 3D views with client scenery, scheduled events (features, vanity changes, live
events, announcements, restarts), vanity files and events, the CDClient browser, the message inspector with saved
captures, chat filter tools, community challenges, live ops, the AI moderator helper, and the server-side changes
they need.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:43 -05:00