Commit Graph

90 Commits

Author SHA1 Message Date
Aaron Kimbrell
d94d84b5cb feat(dashboard): browse every client system info report, per account and across accounts
The account page lists the account's reports as a table and the Client System Info page lists every report (or each account's newest), sortable and searchable by account name or video card; a row opens every field. Values that say little about the real system are marked, and every caveat is a tooltip on the value and column.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 08:06:02 -05:00
Aaron Kimbrell
3a8838d463 feat(capture): replay bundles against a sandbox stack with a headless client
CaptureTool (built next to the servers) replays packet bundles and compares the answers:

- replay: per bundle a fresh sandbox folder with copied server binaries, rewritten settings
  (replay_sandbox=1, a new SQLite file inside the folder, ports from --port on, no dashboard),
  read back before anything starts; sandbox-setup runs inside it to apply migrations and make
  the replay account and the bundle's characters (setup mode); master is started, the stack is
  stopped as one process group and the folder deleted unless kept
- with replay_sandbox=1 every server refuses a database that isn't SQLite, isn't inside its
  own folder, or is replay_live_sqlite_path (Database::Connect); replay-target against a
  running server needs --i-know-this-is-not-a-sandbox
- the fake client splits the recording into connections, logs in and picks the character
  itself when the recording doesn't, fills in the target's account, session key and IDs,
  learns server-made object IDs from replica constructions by LOT, follows the recorded
  timing and waits for the answers a client waits for; the diff pairs answers by name (and
  constructions by LOT) and ignores fields that differ between runs
- import-live converts the 2014 live captures (folders of *_traffic.zip; pcaps and encrypted
  captures are left alone) into bundles, with secrets removed and CREATE_CHARACTER as setup
- anonymise makes local fixtures; docs/CaptureReplay.md describes capture, the bundle format,
  portability rules, the sandbox and the replay

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:34:34 -05:00
Aaron Kimbrell
332bc04ce8 feat(capture): record whole packets of an account, a character or everything on every server
Staff arm a packet capture on the dashboard; master passes MESSAGE_CAPTURE_CONTROL ARM to
every world, auth and chat and arms its own. Each server's PacketCapture tap (dServer receive,
and a send hook in RakPeer::Send so replica constructions are seen too) records into one
preallocated chunk per server and ships sealed chunks through master on the main loop when
capture_flush_bytes or capture_flush_interval_ms is reached; past capture_buffer_max_mb the
oldest chunks are dropped and the dashboard records a gap. Nothing is armed: one flag check.

- targets: an account (from its login; packets before the login are kept per connection
  and added once auth or the world knows whose they are), a character (from when it is
  picked), or everything; up to 8 at once (a bit each in the record mask)
- worlds and auth record their clients' packets and the master link messages of a captured
  player (session keys by name, zone transfers by request, player added/removed, migration);
  chat finds the player in each packet; master records server traffic for everything
- secrets are never recorded: structs that carry them (login request, login response user
  key, world validation session key, session key messages between servers) are read,
  blanked and written again before recording; auth keeps only the handshake and login
- PacketDecoder: a registry by service and message id names every packet and decodes the
  registered structs; CaptureBundle is the file format (DLUBNDL1, metadata, records);
  CaptureTools orders records on one timeline, pulls movement out, makes bundles portable
  or anonymous and diffs replays
- the dashboard keeps packet captures in message_capture_sessions (capture_kind 1) and
  their packets in a file under capture_dir, one write per batch; arming is audited
- MESSAGE_CAPTURE_CONTROL/DATA only gain appended enum values and trailing fields

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:34:34 -05:00
Aaron Kimbrell
d4e9423264 feat(servers): time the main loops' phases and the known heavy spots
Frames and phase scopes in the world, auth, chat and UGC loops (packets per type, entities, physics, ghosting, replica, spawners, log flush, saves, web requests by route). Scopes at LoadPlayer, CreateEntity, each component's construction, InventoryComponent::LoadXml, script timers and a world's zone load; game database queries in the query helpers; CDClient statements timed through sqlite3_trace_v2 as CDClient <table> (CppSQLite3DB gets a handle accessor). Task 96.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
7bc41f2246 feat(db): client system info as reported by the client
client_sysinfo (migrations mysql 102, sqlite 85) keeps the system description
each account's client sent at login, exactly as sent, plus the physical memory
read from it. While nothing but the memory in use changes, the account's newest
row gets the new time and one more login; otherwise a new row starts. Log
pruning deletes rows not seen for a while (eLog::CLIENT_SYSINFO) and deleting
an account deletes its rows.

Tests on SQLite alone (dDatabaseSqliteTests) and in the MySQL parity tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:09:14 -05:00
Aaron Kimbrell
af54ca282f feat(db): chat log guild, team and filter result; chat flags
chat_log gets guild_id, team_id and filtered (migrations mysql 100, sqlite 83),
and the chat log queries take a time range, a guild, a team, a conversation
between two characters and paging back by id. Whispers are their own visibility
switch (includeWhispers), apart from team and guild chat (includePrivate).
Whisper partners and teams that talked are listed with counts.

chat_flags, chat_flag_messages and chat_flag_events (mysql 101, sqlite 84) hold
flagged chat with a copy of the messages around it, its status, note, linked
player report and history. Open flags are in the dashboard snapshot.

Tests on SQLite alone (dDatabaseSqliteTests) and in the MySQL parity tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:23:22 -05:00
Aaron Kimbrell
ef3c74f715 feat(ugc): processing options per make, recorded per model with their times
Staff can make models again with other processing options than the UGC
settings' (ray backend, hidden-face method, denoising) and every make records
what made it, for comparing the options.

- migration mysql 99 / sqlite 82 (ugc_process_options): ugc.process_options
  (picked for the next make, cleared once made), ugc.made_options (what made
  the current files) and ugc_process_runs (every successful make: options,
  wall and CPU time, hidden faces', occlusion's and icon's time, bricks,
  triangles before and after)
- IUgc: ResetUgcModelProcessing and ResetPropertyUgcModelProcessing take the
  options; PendingModel carries them; RecordUgcModelRun, GetUgcRunSummaries;
  list entries have madeOptions and processOptions
- the UGC server applies a job's options over its settings and records the run
- /api/ugc/reprocess takes options ("embree fast oidn"); /api/ugc/options
  lists the choices, the settings' defaults and averages per combination
- /reprocessproperty [builtin|embree|hiprt] [toolbox|fast] [off|oidn], any
  order, all optional

Check: run the migration on MySQL and SQLite; /reprocessproperty embree fast
on a property, then the models' made_options and ugc_process_runs rows;
/api/ugc/options.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:29:08 -05:00
Aaron Kimbrell
33402425e4 feat(dashboard): pending guild names count on the Review Queue and Guilds badges
The dashboard snapshot counts guilds whose name waits for moderation; the moderation counts (WebSocket and
/api/moderation/counts) include it, the Review Queue badge adds it for staff with guilds_manage, and the Guilds menu
entry has its own badge. Check: create a guild with a name off the allow list: both badges go up; approve it: they
go down.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:46:34 -05:00
Aaron Kimbrell
431eddd5bd feat(dashboard): guilds page, guild name review and guild chat in the chat log
A Guilds page (Moderation, permission guilds_manage, GM 5 by default): every guild with its name status, member count,
leader and age, a pending-only switch, and per guild its members (with ranks) and history (guild_events). Staff can
approve a guild name that waits for review, reject it (the guild becomes "Guild <id>"), rename a guild (same name rules
as the game, unique without regard to case), remove a member (a leader's guild goes to the next member, the last
member's guild is deleted) and disband a guild. Every change is audited and added to the guild's history, and the chat
server is asked (GUILD_CHANGED through master) to tell the online members. Pending guild names also show in the Review
Queue. Guild chat ("guild" in the chat log) counts as private chat like whispers and team chat (chat_private), with its
own channel filter and Prometheus counter.

Check on the dashboard: /guilds lists a guild made in game; approve/reject/rename/remove/disband update the in-game guild
window and name billboard of online members; the Review Queue shows a guild whose name waits for review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 06:48:40 -05:00
Aaron Kimbrell
88ebdf685e feat(db): guilds, guild_members, guild_invites and guild_events
Four tables (MySQL migration 97, SQLite 80) and the IGuilds interface with MySQL, SQLite and TestSQL implementations,
for the guild system (docs/Guilds.md). A guild has a name (unique without regard to case), a name status like
pet_names (1 waiting for moderation, 2 approved), its founder and when it was made. A character is in at most one guild
(guild_members, with its rank: 1 leader, 2 officer, 3 veteran, 4 recruit, the client's names, and when it joined) and
has at most one pending invite (guild_invites). guild_events is each guild's history and outlives the guild. Deleting a
character removes its membership and the invites to and from it. Nothing uses the tables yet.

Check: both migrations run on a fresh and an existing database; DatabaseParityTests Guilds passes against MariaDB
(DLU_TEST_MYSQL_HOST) and SQLite gives the same results. Nothing to check in game.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 06:48:40 -05:00
Aaron Kimbrell
821b7c8767 feat(dashboard): permission grants count in every dashboard permission check
What someone may do on the dashboard is now their GM level's permissions plus the grants on their account, minus its
denies (PermissionGrants.h). A deny beats a grant; denies never apply to GM 9, and settings and permissions_manage stay
GM 9 only. The account's grants are read with every request (like its GM level), so a change applies at once, and
they are passed through every check: RouteUtils::Can, CanViewCharacter, the rank rules (self_* and manage_equal_rank),
routes guarded by a permission, the templates' `can`, the API documentation, API access, API key scopes (a key never
does more than its owner may now) and WebSocket subscriptions.

New permission grants_manage (GM 9 by default) and the API to manage grants: GET /api/grants/catalog, GET /api/grants,
POST /api/grants, POST /api/grants/:id/remove. Nobody grants or takes away what they don't hold themselves (a
permission, every permission of a group, a command they may use, every command up to their own GM level), and only on
accounts the rank rules let them manage (their own with self_moderation). Commands with a fixed level or a floor
above GM 1 (/execute) can't be granted. Every change goes in the audit log (grant_permission, deny_permission,
remove_grant). Also: the Showcase gate and the traffic subscription now check their permission by name.

Check: grant a GM 2 account accounts_ban (it can ban, and the Ban button shows); deny a GM 8 account accounts_view (the
accounts list is refused); give an expiry a minute ahead and see it stop; try to grant a permission your account
doesn't have (refused); dWebTests PermissionGrantsTests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 01:16:38 -05:00
Aaron Kimbrell
c575eba4e7 feat(db): permission_grants, permissions and commands given to one account or character
A new table, permission_grants (MySQL migration 96, SQLite 79), and the IPermissionGrants interface with MySQL, SQLite
and TestSQL implementations. A row grants (or with deny, takes away) a dashboard permission, a slash command, a
permission category or every command up to a GM level, for one account or one character, with an optional expiry, who
granted it and when, and a note. Rows are never deleted: removing one sets revoked_at/revoked_by, so the table is also
the history. Nothing reads it yet.

Check: both migrations run on a fresh and an existing database; DatabaseParityTests PermissionGrants passes against
MariaDB (DLU_TEST_MYSQL_HOST) and SQLite gives the same results.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 01:16:37 -05:00
Aaron Kimbrell
180585f9c6 feat(mail): mail history query for the dashboard
IMail::GetMailHistory and CountMailHistory return mail rows as stored,
deleted mail included, newest first, with the sender's and receiver's
accounts. Filters: character (sent or received), account, text in the
subject, body or a name (LIKE wildcards matched literally), state
(unread, read, attachment waiting, claimed, deleted) and whether to
include deleted mail. MySQL and SQLite share the SQL (MailSql.h).

Check: parity test ParitySeeded.Mail (GetMailHistory).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 00:59:49 -05:00
Aaron Kimbrell
acb0e03a74 feat(mail): keep mail a player deletes, marked deleted, for staff
Deleting mail in game now sets mail.deleted_at (migrations mysql 95,
sqlite 78) instead of removing the row. Every read for the game skips
deleted mail: the mailbox, a single mail (claim, read, delete), the
unread count, the economy scan of waiting attachments and the UGC
lookup of mailed models. Deleting a character still removes its mail.

Check: delete a mail in game; it leaves the mailbox, the unread count
drops, and the row is still in the mail table with deleted_at set.
Parity test: ParitySeeded.Mail (DeleteMail).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 00:59:49 -05:00
Aaron Kimbrell
b0dab03807 chore(dashboard): remove the Maintenance page
The page held one-off repairs from Nexus Dashboard (approve known pet
names, find/delete orphaned pet names, remove every buff, fix property
clone IDs, list mission rewards without a commendation price). Removed
the page, its sidebar entry, its /api/maintenance/* routes, the
`maintenance` permission and the two database calls only it used
(GetAllPetNames, FixPropertyCloneIds). The scheduled tasks (lift expired
bans, fill in pet owners, approve known pet names) and property model
import/removal stay.

Check: the Admin sidebar group has no Maintenance link; /maintenance is
a 404; the Tasks page still lists "Approve known pet names" and "Fill in
pet owners"; property import still works; the Permissions page no
longer lists "Data maintenance".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 23:44:44 -05:00
Aaron Kimbrell
4679edd2b2 feat(ugc): processing totals on the UGC page; no shared icon preset for player models
- IUgc::GetUgcProcessTotals sums what the UGC server made, for models and
  for car and rocket builds: count, time spent (and how many are timed),
  CPU time, average and slowest, average and most memory (estimate),
  bricks, triangles and triangles saved (models whose count before hidden
  face removal is known). /api/ugc returns them as totals and the UGC page
  shows a card for each kind. Parity tested.
- Player models no longer have a shared icon preset: every one is a
  different size and shape, so its icon is fitted to it from the settings.
  The UGC server ignores a kind:model preset, the dashboard refuses to save
  one, and the icon editor hides the type buttons for models. One model's
  own icon values still work. Car and rocket build types keep theirs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:25 -05:00
Aaron Kimbrell
9e4f2818c3 feat(ugc): staff reprocessing of a property goes to the front of the queue
/reprocessproperty and the dashboard's Reprocess all models mark the
property's models as priority (ugc.priority, migrations mysql 94 and sqlite
77). The UGC server takes priority models first, polls them even when its
queue is full, and puts them at its front, as it does cars and rockets. The
flag clears once a model is made.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:21 -05:00
Aaron Kimbrell
bfa46408cb feat(db): properties_contents.placed_by, the character who placed a model
NULL is the owner (every model placed before this, and dashboard imports).
Read by GetPropertyModels and GetModel, written by InsertNewPropertyModel on
MySQL and SQLite, with a parity test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:20 -05:00
Aaron Kimbrell
6c45d4abdb feat(dashboard): reprocess or remove every model on a property
The property page gets a Reprocess all models button (ugc_manage): every
model placed on the property goes back to the UGC server's queue, made
again with the current settings (/api/ugc/reprocess with {property}).
Import models gets Remove all models (properties_import): deletes every
model placed on the property after the property id is typed in, refused
while the property is loaded in a world, as import is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:18 -05:00
Aaron Kimbrell
f252c6ff26 feat(ugc): how many triangles each model saved, on the UGC page
The UGC server stores a model's most detailed level's triangles before it
removed the faces that can't be seen (ugc.triangle_count_before;
migrations sqlite 75, mysql 92), from stats.json when it makes a model
and, once, for the models made before. The UGC page's models list has a
Saved column (the share and number of triangles removed, before/after in
its tooltip), sortable by the share (sort=savings), and the gallery can
sort by most triangles saved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:16 -05:00
Aaron Kimbrell
c939a69240 feat(dashboard): UGC list shows a model's given name, and Took/CPU/RAM columns
The models list's Took, CPU and RAM (est.) are separate columns, each
sortable (sort=slowest|cpu|memory). The File column shows the name a
player gave the model where it is placed, with the upload's extension
(the upload's file name is its tooltip), and the name sort uses it
(case-insensitive, so SQLite and MySQL agree).

The config layer test no longer assumes the build's sharedconfig.ini has
no mysql_host; it checks that the database-supplied value isn't used.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:16 -05:00
Aaron Kimbrell
264abfc0f3 feat(dashboard): UGC list sorts by when a model was made and by how long it took
The models list has separate Made (when) and Took (the time, CPU and
estimated memory) columns, each sortable in either direction; /api/ugc
takes sort=made (processed_at) besides sort=slowest, and the gallery can
sort by recently made.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:16 -05:00
Aaron Kimbrell
cd81cf4c74 feat(ugc): each make's CPU time and memory; durations in readable units
The UGC server records, with each successful make's time, the worker
thread's CPU time for it and the memory it estimated the job needs (the
figure its memory budget counts; not a measurement)
(process_cpu_ms and process_memory_kb on ugc and ugc_modular_build;
migrations sqlite 74, mysql 91). The UGC page shows them with the time
and duration ("took 12.4 s, CPU 11.9 s, ~96 MB RAM (est.)"), and
durations use the largest units that fit, up to days.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:15 -05:00
Aaron Kimbrell
90777e5c9d feat(ugc): when each creation was made and how long it took, on the UGC page
The UGC server records how long each successful make took (process_ms on
ugc and ugc_modular_build; migrations sqlite 73, mysql 90). The UGC page
shows it with the time it was made: in the models list's Made column
(sortable, slowest first), in the tiles' tooltips, in the item preview
and in an assembly's References. The gallery can sort by slowest to
make too. Makes from before this are shown without a duration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:15 -05:00
Aaron Kimbrell
79235702b3 feat(ugc): answer the client's UGC manifest requests without 3D services
With UGCUSE3DSERVICES=7:0 (the client's default) the client asks its world for
a blueprint file's MD5 and size (REQUEST_UGC_MANIFEST_INFO, world 27) and then
downloads BrickModels/UserMade/<id % 1000>/<id>.<ext>.sd0. Layouts checked in
the 1.10.64 client: the request is a u64 blueprint id and a u8 resource type;
the answer (UGC_MANIFEST_RESPONSE, client 60) repeats them and adds a u8 valid,
the u32 size and the 16 byte MD5 of the inflated file, 37 bytes after the 0x53
exactly or the client drops it.

- dNet: WorldPackets::RequestUgcManifestInfo, ClientPackets::UgcManifestResponse
  (eUgcResourceType), with byte tests against the client's layouts.
- Database: ugc_file_checksums (per model or module combination and file) and
  ugc_modular_build.combination_id (migrations 89 / 72), GetUgcFileChecksum
  looks a blueprint up as a model, else as a build through its combination.
- UGC server: every download is also written as .sd0 (Sd0::Compress); workers
  hand the checksums back and the main thread stores them; old items get their
  sd0 icon and checksum, and builds their combination id, once at start-up, a
  few per tick; serves <dir>/BrickModels/UserMade/<bucket>/<id>.<ext>.sd0 under
  client_path, /<folder>/UserBrickModels and the root (.hkx 404).
- World: UgcManifest answers on the main thread with one indexed query per
  request; files not made yet are answered when they are (looked at again
  every 5 seconds), and a model in its quiet period is made right away. No
  worker threads, HTTP or file reads in the world.

Off by default (ugc_manifest=0): checked in game, the client then downloads
from http://127.0.0.1:80/lwoclient/UserBrickModels/ whatever its boot.cfg says
and logs the player out when it can't connect, so icons need the UGC server on
port 80 of each player's machine. docs/UgcServer.md has the details.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:14 -05:00
Aaron Kimbrell
453f728b5e feat(dashboard): UGC List view and References as sortable DataTables
The UGC page's List view (models and car/rocket assemblies) and an
assembly's References panel are server-side DataTables like the rest of
the dashboard: sort by clicking a column heading, in either direction,
with the page length and sort remembered per user. They still read
/api/ugc and /api/ugc/assembly/builds, so the prefix search and filters
above the list keep working; the gallery keeps its own pager and sort.

/api/ugc takes reverse=1 for a sort's other direction (the model list's
SQL order and the assembly sort both flip), and the references endpoint
sorts by id, owner, account or state. Tests cover the reversed orders.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:13 -05:00
Aaron Kimbrell
4f3983fb98 feat(dashboard): UGC lists paged with totals, prefix search and sorting; car and rocket assemblies
IUgcLookup::ListUgc pages either kind with the UGC search's matching, a state
and a sort, and counts the matches (MySQL and SQLite share the SQL). The UGC
server records each model's bricks and triangles (new ugc columns) so models
can be sorted by size. Cars and rockets are listed as assemblies, one per
combination of modules, with their build type, module names and how many
builds use them; filters for type, state, module and owner. Each assembly's
builds (with where they are) are paged, and a build's assembly can be looked
up for links.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:12 -05:00
Aaron Kimbrell
5de59b99ef feat(ugc): models with no bricks are "empty", not failed
A model with no bricks has nothing to make: it gets its own state
(is_optimized = 3), isn't counted as a failure or retried, shows as Empty
on the UGC page (with its own filter and count), in the status and in
Prometheus, and its downloads answer 404 like HKX. State names come from
the enum (magic_enum). Migrations dlu/mysql/87 and dlu/sqlite/70 move the
rows that failed only because they had no bricks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:10 -05:00
Aaron Kimbrell
1a2758feab feat(ugc): icon light matched to the game, debounce, shared car icons, purge, icon editor
Brighter icons: a world light, a fill from the camera, a highlight, exposure
and contrast; with the defaults the icons' mean luminance matches the game's
own model icons (118 against 120 on a scratch set). Every icon parameter is
listed once (UgcIconParams: key, setting, range, default); the settings,
the dashboard's settings entries and the icon editor come from that list.
Presets per kind (player models, each car or rocket build type from
ModularBuildComponent) and overrides per model or module combination are in
ugc_icon_settings.

Saved models wait ugc_debounce_seconds (sharedconfig) after the owner's last
save before they're made (ugc.process_after); a client asking for one, the
owner leaving the world or a reset ends the wait.

Cars and rockets: one icon per combination of modules (sorted LOTs), shared
by every build of it; builds of a combination made already are marked made
right away, the client's per-blueprint downloads serve the shared files.

The dashboard can delete one item's files, purge by filter or all, preview
icons with any values on the UGC server (/admin routes, master password),
save presets and overrides, and draw a kind's icons again (icons only).

Migrations dlu/mysql/86 and dlu/sqlite/69. Not done yet: the dashboard
editor's lighting controls in the page script (routes are there), docs for
it, the empty-model state, /ugc?item= links, the shared fetch helper; the
storage size and property loading bugs are next.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:09 -05:00
Aaron Kimbrell
7534f48de9 feat(database): look up players' creations and where they are
IUgcLookup: search ugc and ugc_modular_build by id, creator, property, model
name or LOT, and find where a creation is placed or mailed. The SQL is shared
by MySQL and SQLite (UgcLookupSql.h).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:08 -05:00
Aaron Kimbrell
95158f69b6 feat(dashboard): UGC gallery and viewer, reached through the dashboard
The /ugc page no longer needs the browser to reach the UGC server: the
dashboard fetches its status (/api/ugc/server/status), the files it made
(/api/ugc/files/<kind>/<id>/<file>) and its NIFs converted for the 3D view
(/api/ugc/mesh/<id>, NifFile like the scenery) from ugc_internal_url
(default http://127.0.0.1:2008) with libcurl on the worker threads, keeping
small answers briefly. ugc_public_url is only an "open on the UGC server"
link now.

The page gets an icon gallery beside the list, filtered by kind, state and a
search by id or owner (GetUgcProcessList/GetModularBuildProcessList take a
search), and a viewer: the generated NIF in 3D at any LOD, now or before it
was made again, with wireframe, vertex color and baked lighting switches, the
LXFML beside it, the icon now and before, and the stats with triangles before
and after hidden faces were removed. The status box shows CPU, memory, the
jobs' memory estimate with their limits, and throttling. The settings page
lists the UGC server's new settings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:07 -05:00
Aaron Kimbrell
e669a4fa7e feat(dashboard): track the UGC server like auth and chat
Online state with the time it came up, down/up alerts, a UGC column in the
health history, /api/servers (every server with its process memory and CPU)
and /api/servers/ugc, and UGC gauges for Prometheus.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:05 -05:00
Aaron Kimbrell
8001070501 feat(database): dashboard_api_keys table
Hashed API keys with scope, restrictions, limits, expiry and batched
usage counters, for MySQL and SQLite, with test stubs and parity tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:03 -05:00
Aaron Kimbrell
01d10f4401 feat(database): server_traffic table for per-minute traffic rollups
One row per server and minute (packets, bytes, resends, HTTP requests, errors,
bytes and latency percentiles), written in batches and read summed into
buckets for the longer chart ranges.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:01 -05:00
Aaron Kimbrell
6c414cec48 feat(dashboard): check uploaded character XML before storing it
The XML upload is the one place hand-written XML reaches the game, whose
load path trusts the XML because the server writes it itself. Instead of
making the game skip bad data at load, the upload is checked against
what the load path assumes and refused (400, with the problems) when the
game couldn't load it: required elements, attributes that must parse
(flags read with std::stoul/stoull), required item and mission fields,
known inventory types, mission states and character versions, items and
missions that exist in the CDClient, unique item IDs and slots, and the
acct attribute matching the owner.

Suspicious but loadable content is returned as warnings that need
confirm=true (409 otherwise): contraband (same matching as the world,
now shared in ContrabandRules.h), stacks above the stack size, coins,
level or u-score out of reach, a GM level above the account's.
Contraband marked flag-and-remove is removed only if the uploader asks;
once stored, findings are flagged (CONTRABAND) and audited. The XML
editor shows the findings and offers "Save anyway". Related: issue 1332.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:59 -05:00
Aaron Kimbrell
9256b31b23 feat(db): bbb_autosave table for the client's BBB autosave
The client sends SetBBBAutosave (996) with the model being built every five
minutes, before an AFK kick and before quitting, and expects the server to
rebuild an unfinished model later (RebuildBBBAutosaveMsg). This keeps the
last one per character with the model items that were in the BBB inventory
at the time. MySQL 82 / SQLite 65; parity test included.

Refs #1632

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:58 -05:00
Aaron Kimbrell
abf1cc1d80 feat: UGC server that makes and serves player models' meshes and icons
A new server (dUgcServer, started by master with enable_ugc_server=1) that
takes unprocessed ugc and ugc_modular_build rows from the database and makes
what the client downloads with UGCUSE3DSERVICES: an optimized NIF (hidden
faces removed, ambient occlusion baked into vertex colors) and a 128px DDS
icon for player models, rendered by a software rasterizer from the client's
LDD brick primitives, and icons for cars and rockets assembled from their
modules per ModularBuildComponent/ModuleComponent. It serves them, with the
models' LXFML, over HTTP in the client's UGCC<dc>/3DOPTIMIZED and
IMAGE128DDS layout with .gz and .checksum files, and keeps its folder under
a size cap.

Processing state lives in ugc.is_optimized plus new processed_at,
process_attempts and process_error columns (and the same on
ugc_modular_build). ServiceType::UGC is appended. NifFile moves to dCommon
and records named node transforms for the modules' attach points.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:57 -05:00
Aaron Kimbrell
9682556128 feat: store each named pet's LOT in pet_names
pet_names gets a pet_lot column (mysql 80, sqlite 63). The world writes
it whenever it saves a pet name, from the pet entity's LOT, and fills it
in for older rows when the owner loads into a world (from the pets the
game loads for that character, only where it is still missing).

The dashboard's pet name tables read pet_lot instead of scanning the
owner's character XML; pets without it yet show as Unknown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:57 -05:00
Aaron Kimbrell
d95eab1212 feat: property reputation from visitors, resistant to farming
Property worlds now give their property reputation for the time other people spend on it, which
fills the property lists and the news screen's Today's Top Properties. Visitors are counted per
account; the owner's account, accounts linked to it and staff don't count. A visit earns nothing
for the first property_reputation_min_visit seconds (WorldConfig's propertyReputationDelay), then
each active minute (the visitor moved) earns reputationPerMinute times a multiplier, for a capped
number of minutes per visit. Repeat visitors earn less the more recent days they already gave
reputation, and each visitor and each property have a daily cap. Every parameter is a setting;
what each account gave each property per day is kept in property_reputation_visits. The rules are
pure functions with unit tests.

Fixes #636
Fixes #637

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:52 -05:00
Aaron Kimbrell
680615ba9f feat: optional property rent
Off unless property_rent_enabled is on. Each property world's rent comes from its PropertyTemplate
row (minimumPrice every rentDuration x durationType; Block Yard is free), unless the new Property
Rent dashboard page sets another price or period (property_rent_manage). Rent is taken from the
owner's coins shortly after their character loads, with a mail receipt; unpaid rent is mailed and,
after property_rent_grace_days, makes the property private until it is paid, like live. The
property management component refuses public or best friends privacy while rent is overdue and a
property world that loads overdue makes itself private. Property game messages are unchanged.

Fixes #943

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:52 -05:00
Aaron Kimbrell
b2403b09ea feat: contraband list with flagging and optional removal
Staff list contraband items on a new dashboard page (item search, reason, flag or flag and remove;
contraband_manage to edit, reports_view to see). World servers check every inventory when a
character loads and every item a player receives: each find is an economy flag of the new kind
Contraband, shown with the other flags and in the character's related data. Items marked for
removal are taken away, with a character snapshot kept first so they can be given back, an audit
entry and a mail or chat message telling the player why. Staff are skipped unless
contraband_ignore_staff is off. Worlds reload the list when it changes (RELOAD_CONTRABAND, added
at the end of ePlayerAction).

Fixes #1563

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:52 -05:00
Aaron Kimbrell
341ce89a6a feat: refuse stale character saves with a save generation
Every character gets a save generation in charxml. A world bumps it when it loads the character for
play, and every save from that world only goes through while the stored generation is still the
one it loaded or last saved (and moves it on). Dashboard edits, restores and maintenance writes bump
it too. A world that lost the character to another world (a disconnect noticed late, a zone
transfer, an instance migration) or to a dashboard edit can no longer overwrite the newer data: the
save is refused, logged and audited as stale_save_refused, the world stops saving that character
and a player still connected to it is disconnected with the save failure reason so they reload.

Fixes #639

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:51 -05:00
Aaron Kimbrell
6b9310e513 perf(dashboard): economy reports on a large ledger
- Leave staff out of the currency and U-score reports by excluding the few staff characters,
  instead of joining every ledger row to its character and account.
- Top earners: total per character first and look up names for the top rows only.
- The places list and the activity report read map events for every kind in one query
  (GetMapZonesAllKinds) instead of one query per kind.

With about 1M currency rows and 650k map event rows (90 days) on MariaDB: currency 1.8 s to
0.8 s, top earners 2.4 s to 0.75 s, places 2.6 s to 0.5 s, activity 2.6 s to 1.7 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:47 -05:00
Aaron Kimbrell
99aecc6b74 fix(dashboard): activity log table on large MySQL databases
- Read activity and map_id as signed: a row with map_id -1 made the page fail with a server error
  when sorted by map.
- A search counts and pages through the characters it matches (index on character_id) instead
  of joining every row of the log to a name; when the matches are a large part of the log the
  page still walks the log in order.
- Sorted by character name, only the row ids are sorted and the page's rows are read after.

On a log of about 800k rows this takes searches with few matches from about 1 s to 0.1 s and
name sorts from about 3 s to 0.5 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:47 -05:00
Aaron Kimbrell
e213a7aeff feat: web dashboard and playground work
The NexusDashboard-parity dashboard (dDashboardServer) and everything built on it on the experimental branch:
accounts, characters, properties and moderation tools, permissions shared with in-game slash commands, economy
reports, World 3D and property 3D views with client scenery, scheduled events (features, vanity changes, live
events, announcements, restarts), vanity files and events, the CDClient browser, the message inspector with saved
captures, chat filter tools, community challenges, live ops, the AI moderator helper, and the server-side changes
they need.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:43 -05:00
David Markowitz
a156a8fcba fix: security vulnerabilities (#1980)
* fix: security vulnerabilities

Tested that all functions related to the touched files work

will test sqlite on a CI build

* fix failing test

* ai feedback

* add buffer size checking

* use c_str

* dont log session key

* Try this for a mac definition

* be quiet apple
2026-06-07 20:59:11 -07:00
David Markowitz
247576e101 fix: use copy ellision (#1963)
* use copy ellision

tested that the server still starts

* Update dDatabase/GameDatabase/MySQL/MySQLDatabase.h

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-31 15:35:28 -07:00
David Markowitz
8dfdca7fbd feat: add mission progression for behaviors (#1962)
* feat: add mission progression for behaviors

* Add const to ptr

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-31 13:02:23 -07:00
David Markowitz
76c2f380bf feat: re-write persistent object ID tracker (#1888)
* feat: re-write persistent object ID tracker

Features:
- Remove random objectIDs entirely
- Replace random objectIDs with persistentIDs
- Remove the need to contact the MASTER server for a persistent ID
- Add persistent ID logic to WorldServers that use transactions to guarantee unique IDs no matter when they are generated
- Default character xml version to be the most recent one

Fixes:
- Return optional from GetModel (and check for nullopt where it may exist)
- Regenerate inventory item ids on first login to be unique item IDs (fixes all those random IDs

Pet IDs and subkeys are left alone and are assumed to be reserved (checks are there to prevent this)
There is also duplicate check logic in place for properties and UGC/Models

* Update comment and log

* fix: sqlite transaction bug

* fix colliding temp item ids

temp items should not be saved. would cause issues between worlds as experienced before this commit
2025-09-29 08:54:37 -05:00
David Markowitz
b5a3cc9187 fix: Extend saved ugc id to 64 bits (#1885)
Tetsed that ugc models are saved and loaded with the correct bits
2025-09-24 06:01:46 -05:00