CaptureTool (built next to the servers) replays packet bundles and compares the answers:
- replay: per bundle a fresh sandbox folder with copied server binaries, rewritten settings
(replay_sandbox=1, a new SQLite file inside the folder, ports from --port on, no dashboard),
read back before anything starts; sandbox-setup runs inside it to apply migrations and make
the replay account and the bundle's characters (setup mode); master is started, the stack is
stopped as one process group and the folder deleted unless kept
- with replay_sandbox=1 every server refuses a database that isn't SQLite, isn't inside its
own folder, or is replay_live_sqlite_path (Database::Connect); replay-target against a
running server needs --i-know-this-is-not-a-sandbox
- the fake client splits the recording into connections, logs in and picks the character
itself when the recording doesn't, fills in the target's account, session key and IDs,
learns server-made object IDs from replica constructions by LOT, follows the recorded
timing and waits for the answers a client waits for; the diff pairs answers by name (and
constructions by LOT) and ignores fields that differ between runs
- import-live converts the 2014 live captures (folders of *_traffic.zip; pcaps and encrypted
captures are left alone) into bundles, with secrets removed and CREATE_CHARACTER as setup
- anonymise makes local fixtures; docs/CaptureReplay.md describes capture, the bundle format,
portability rules, the sandbox and the replay
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Staff arm a packet capture on the dashboard; master passes MESSAGE_CAPTURE_CONTROL ARM to
every world, auth and chat and arms its own. Each server's PacketCapture tap (dServer receive,
and a send hook in RakPeer::Send so replica constructions are seen too) records into one
preallocated chunk per server and ships sealed chunks through master on the main loop when
capture_flush_bytes or capture_flush_interval_ms is reached; past capture_buffer_max_mb the
oldest chunks are dropped and the dashboard records a gap. Nothing is armed: one flag check.
- targets: an account (from its login; packets before the login are kept per connection
and added once auth or the world knows whose they are), a character (from when it is
picked), or everything; up to 8 at once (a bit each in the record mask)
- worlds and auth record their clients' packets and the master link messages of a captured
player (session keys by name, zone transfers by request, player added/removed, migration);
chat finds the player in each packet; master records server traffic for everything
- secrets are never recorded: structs that carry them (login request, login response user
key, world validation session key, session key messages between servers) are read,
blanked and written again before recording; auth keeps only the handshake and login
- PacketDecoder: a registry by service and message id names every packet and decodes the
registered structs; CaptureBundle is the file format (DLUBNDL1, metadata, records);
CaptureTools orders records on one timeline, pulls movement out, makes bundles portable
or anonymous and diffs replays
- the dashboard keeps packet captures in message_capture_sessions (capture_kind 1) and
their packets in a file under capture_dir, one write per batch; arming is audited
- MESSAGE_CAPTURE_CONTROL/DATA only gain appended enum values and trailing fields
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Frames and phase scopes in the world, auth, chat and UGC loops (packets per type, entities, physics, ghosting, replica, spawners, log flush, saves, web requests by route). Scopes at LoadPlayer, CreateEntity, each component's construction, InventoryComponent::LoadXml, script timers and a world's zone load; game database queries in the query helpers; CDClient statements timed through sqlite3_trace_v2 as CDClient <table> (CppSQLite3DB gets a handle accessor). Task 96.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The world and master servers pass the client's res/cdclient.fdb to
CDClientManager. When it opens, these three tables, all looked up by
their first column, find rows through the fdb's buckets instead of each
process caching the whole table: ComponentsRegistry keeps nothing,
ItemComponent and Objects keep only the entries asked for (their API
returns references). Ids whose rows CDServer.sqlite changes are loaded
from SQLite at startup and win. Without an fdb (or with one whose
columns don't match) the tables load from CDServer.sqlite as before.
ItemComponent and Objects now fill entries from one template for both
sources instead of copies of the same field list.
Tests cover the SQLite changes on top of the fdb, the no-fdb and
unmapped paths, and, when DLU_CLIENT_RES points at a client, every id of
the three tables through the fdb against CDServer.sqlite.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CDFdb opens cdclient.fdb once per process and hands out a table when its
columns match the CDServer.sqlite table of the same name. CDServer.sqlite
stays the source of truth: the CDServer migrations change a few rows, so
FindChangedKeys compares both files row by row (a hash per row, summed
per key) and returns the keys that differ, for the tables to read from
SQLite. RowFields reads an fdb row with CppSQLite3Query's accessors and
defaults, so a table fills its entries from either file with one piece
of code.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DestructibleComponent.attack_priority is loaded onto the DestroyableComponent as a signed value. The client's
LWODestroyableComponent (LoadDataFromTemplate, 0x00c9f900) starts it at 1 and keeps 1 when the column is empty, and
answers GetAttackPriority with it; the server now does the same so TacArcs can order their targets by it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every entity and every inventory item looks these up, and an id not seen yet was a full scan of the
unindexed table, so a character holding about 3000 different items took a minute to load (the client
waited at 35%). Loading both tables once at startup costs a few MB per server.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
client_sysinfo (migrations mysql 102, sqlite 85) keeps the system description
each account's client sent at login, exactly as sent, plus the physical memory
read from it. While nothing but the memory in use changes, the account's newest
row gets the new time and one more login; otherwise a new row starts. Log
pruning deletes rows not seen for a while (eLog::CLIENT_SYSINFO) and deleting
an account deletes its rows.
Tests on SQLite alone (dDatabaseSqliteTests) and in the MySQL parity tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chat_log gets guild_id, team_id and filtered (migrations mysql 100, sqlite 83),
and the chat log queries take a time range, a guild, a team, a conversation
between two characters and paging back by id. Whispers are their own visibility
switch (includeWhispers), apart from team and guild chat (includePrivate).
Whisper partners and teams that talked are listed with counts.
chat_flags, chat_flag_messages and chat_flag_events (mysql 101, sqlite 84) hold
flagged chat with a copy of the messages around it, its status, note, linked
player report and history. Open flags are in the dashboard snapshot.
Tests on SQLite alone (dDatabaseSqliteTests) and in the MySQL parity tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Staff can make models again with other processing options than the UGC
settings' (ray backend, hidden-face method, denoising) and every make records
what made it, for comparing the options.
- migration mysql 99 / sqlite 82 (ugc_process_options): ugc.process_options
(picked for the next make, cleared once made), ugc.made_options (what made
the current files) and ugc_process_runs (every successful make: options,
wall and CPU time, hidden faces', occlusion's and icon's time, bricks,
triangles before and after)
- IUgc: ResetUgcModelProcessing and ResetPropertyUgcModelProcessing take the
options; PendingModel carries them; RecordUgcModelRun, GetUgcRunSummaries;
list entries have madeOptions and processOptions
- the UGC server applies a job's options over its settings and records the run
- /api/ugc/reprocess takes options ("embree fast oidn"); /api/ugc/options
lists the choices, the settings' defaults and averages per combination
- /reprocessproperty [builtin|embree|hiprt] [toolbox|fast] [off|oidn], any
order, all optional
Check: run the migration on MySQL and SQLite; /reprocessproperty embree fast
on a property, then the models' made_options and ugc_process_runs rows;
/api/ugc/options.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dashboard snapshot counts guilds whose name waits for moderation; the moderation counts (WebSocket and
/api/moderation/counts) include it, the Review Queue badge adds it for staff with guilds_manage, and the Guilds menu
entry has its own badge. Check: create a guild with a name off the allow list: both badges go up; approve it: they
go down.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Guilds page (Moderation, permission guilds_manage, GM 5 by default): every guild with its name status, member count,
leader and age, a pending-only switch, and per guild its members (with ranks) and history (guild_events). Staff can
approve a guild name that waits for review, reject it (the guild becomes "Guild <id>"), rename a guild (same name rules
as the game, unique without regard to case), remove a member (a leader's guild goes to the next member, the last
member's guild is deleted) and disband a guild. Every change is audited and added to the guild's history, and the chat
server is asked (GUILD_CHANGED through master) to tell the online members. Pending guild names also show in the Review
Queue. Guild chat ("guild" in the chat log) counts as private chat like whispers and team chat (chat_private), with its
own channel filter and Prometheus counter.
Check on the dashboard: /guilds lists a guild made in game; approve/reject/rename/remove/disband update the in-game guild
window and name billboard of online members; the Review Queue shows a guild whose name waits for review.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Four tables (MySQL migration 97, SQLite 80) and the IGuilds interface with MySQL, SQLite and TestSQL implementations,
for the guild system (docs/Guilds.md). A guild has a name (unique without regard to case), a name status like
pet_names (1 waiting for moderation, 2 approved), its founder and when it was made. A character is in at most one guild
(guild_members, with its rank: 1 leader, 2 officer, 3 veteran, 4 recruit, the client's names, and when it joined) and
has at most one pending invite (guild_invites). guild_events is each guild's history and outlives the guild. Deleting a
character removes its membership and the invites to and from it. Nothing uses the tables yet.
Check: both migrations run on a fresh and an existing database; DatabaseParityTests Guilds passes against MariaDB
(DLU_TEST_MYSQL_HOST) and SQLite gives the same results. Nothing to check in game.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cars and rockets built before builds were recorded were saved with
subkey 0 and have no ugc_modular_build row, so the client has no
blueprint id to ask for their icon with. The runtime fix only reaches
characters that load again.
Migrations mysql 98 and sqlite 81 (98/81 because the guilds work takes
97/80) run ModularBuildIdMigration after the SQL: every saved item with
modules (x@ma) and no subkey gets what a new build gets, as the
character-load fix does: a persistent id from object_id_tracker (with
the character bit) as its subkey and a ugc_modular_build row with its
modules and the character as owner. The XML is written with
UpdateCharacterXml. Tried on a copy of a server's database with a
couple of thousand such items: every one got an id and a build row, in
seconds.
Check: after the migration, old cars and rockets show their icons in the
backpack (the UGC server makes them like any build) and still work
(equip, race, launch).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DownloadPropertyData sent the same zonePosition (548, 406, 178) and a
maxBuildHeight of 128 for every property. Both now come from the
property's PropertyTemplate row (zoneX/zoneY/zoneZ, maxBuildHeight), the
same row the template ID, vendor map and spawn name already come from.
A map with no row still sends the old values.
Live rows: Avant Gardens small (1150) keeps 548/406/178 and 128; Avant
Gardens medium (1151) is 428/413/82 with a build height of 256; the
Nimbus Station (1250, 1251), Gnarled Forest (1350) and Forbidden Valley
(1450) properties get their own positions with 128.
Check in game: on the medium Avant Gardens property (1151) models can be
placed higher than before (up to 256), and on the small one (1150) the
limit is unchanged; on the other worlds' properties the property screens
(plaque, news/claim UI) still show and building works as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
EventGating (eventName, date_start, date_end; 8 rows) is only in the
CDClient for the server: the 1.10.64 client has no string for it. The
dates are Unix times (UTC), inclusive: pirateDay 2011-09-19 00:00 to
2011-09-20 23:59:59 (Talk Like a Pirate Day), buildNexusTower 2010-03-09
to 2011-03-14, buildNexusTowerEnd, frostburgh2010_notused and test rows.
The live Lua asked for an event with GetHolidayEvent{eventToCheck =
name}.isValid; the only callers are the Crux Prime random spawners
(L_BASE_RANDOM_SPAWNER.lua, L_AM_ZONE_SERVER.lua), whose only event is
pirateDay. DLU's BaseRandomServer::CheckEvents was a TODO.
- CDEventGatingTable loads the table; HolidayEvents::IsActive(name) is
true while the dates include now, or when event_1..event_8 names the
event (every live date is in 2010/2011, so this is how a server turns
one on; the same settings already switch gatingOnFeature objects).
- BaseRandomServer::CheckEvents: during pirateDay the str and zip areas
use the event loads from the Lua (80%: 5 pirates on each of type1-3,
20%: 5 admirals each; multipliers secA 1, secB 1, secC 1.2 for str).
Inferred: every spawner's Lua put the whole {str, zip} table in its
zones, which its getRandomLoad walks with ipairs and so finds no load;
DLU uses each table for the area it names, as L_AM_ZONE_SERVER.lua's
layout intends.
- sharedconfig.ini says event_N also takes EventGating names.
Check in game: with event_2=pirateDay in sharedconfig.ini, the Crux Prime
areas filled by the str and zip random spawners (spawner networks
em_str_* and em_zip_*) spawn only Maelstrom pirates and admirals; the
other two areas and a server without the setting spawn the usual mix.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CollectibleComponent (79 rows: id, requirement_mission) is only in the
CDClient for the server: the 1.10.64 client has no string for it. DLU
never read it.
What the column holds (1.10.64 CDClient, joined with ComponentsRegistry,
Missions and MissionTasks): for 63 rows it is the mission the collectible
belongs to. Mostly that is the achievement whose collection task
(taskType 3) targets the collectible's LOT (the flags, imagination bricks,
Johnny Thunder collectibles). For a few it is a mission to accept from an
NPC that does not collect the item itself: the four Ninjago dragon relics
(16483-16485) are collected by the hidden achievements 2064-2067 but
their requirement is 2040 (accepted from LOT 13789, "complete 2064-2067").
Other values: -1 and 66666666 (no such mission) on test rows.
So a collectible whose requirement_mission is a mission to accept
(Missions.isMission) now only counts (HasBeenCollected progresses
nothing) while the player has that mission accepted and not handed in
(ACTIVE, READY_TO_COMPLETE or their repeat states). Achievements, missing
missions and rows without one are unchanged. Without this, a player who
had not accepted 2040 could collect the relics early and have 2040
complete as soon as it was accepted. The gate itself is inferred from the
data: the captures show collections but not the live server's check.
The collectible's object report shows the requirement mission.
Check in game: in Ninjago Monastery, before accepting the dragon relic
mission (2040), touch a dragon relic: it does not count; accept 2040 and
collect them: each counts and 2040 completes after the fourth. Flags,
imagination bricks and Johnny Thunder collectibles still count as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What someone may do on the dashboard is now their GM level's permissions plus the grants on their account, minus its
denies (PermissionGrants.h). A deny beats a grant; denies never apply to GM 9, and settings and permissions_manage stay
GM 9 only. The account's grants are read with every request (like its GM level), so a change applies at once, and
they are passed through every check: RouteUtils::Can, CanViewCharacter, the rank rules (self_* and manage_equal_rank),
routes guarded by a permission, the templates' `can`, the API documentation, API access, API key scopes (a key never
does more than its owner may now) and WebSocket subscriptions.
New permission grants_manage (GM 9 by default) and the API to manage grants: GET /api/grants/catalog, GET /api/grants,
POST /api/grants, POST /api/grants/:id/remove. Nobody grants or takes away what they don't hold themselves (a
permission, every permission of a group, a command they may use, every command up to their own GM level), and only on
accounts the rank rules let them manage (their own with self_moderation). Commands with a fixed level or a floor
above GM 1 (/execute) can't be granted. Every change goes in the audit log (grant_permission, deny_permission,
remove_grant). Also: the Showcase gate and the traffic subscription now check their permission by name.
Check: grant a GM 2 account accounts_ban (it can ban, and the Ban button shows); deny a GM 8 account accounts_view (the
accounts list is refused); give an expiry a minute ahead and see it stop; try to grant a permission your account
doesn't have (refused); dWebTests PermissionGrantsTests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new table, permission_grants (MySQL migration 96, SQLite 79), and the IPermissionGrants interface with MySQL, SQLite
and TestSQL implementations. A row grants (or with deny, takes away) a dashboard permission, a slash command, a
permission category or every command up to a GM level, for one account or one character, with an optional expiry, who
granted it and when, and a note. Rows are never deleted: removing one sets revoked_at/revoked_by, so the table is also
the history. Nothing reads it yet.
Check: both migrations run on a fresh and an existing database; DatabaseParityTests PermissionGrants passes against
MariaDB (DLU_TEST_MYSQL_HOST) and SQLite gives the same results.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
IMail::GetMailHistory and CountMailHistory return mail rows as stored,
deleted mail included, newest first, with the sender's and receiver's
accounts. Filters: character (sent or received), account, text in the
subject, body or a name (LIKE wildcards matched literally), state
(unread, read, attachment waiting, claimed, deleted) and whether to
include deleted mail. MySQL and SQLite share the SQL (MailSql.h).
Check: parity test ParitySeeded.Mail (GetMailHistory).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Deleting mail in game now sets mail.deleted_at (migrations mysql 95,
sqlite 78) instead of removing the row. Every read for the game skips
deleted mail: the mailbox, a single mail (claim, read, delete), the
unread count, the economy scan of waiting attachments and the UGC
lookup of mailed models. Deleting a character still removes its mail.
Check: delete a mail in game; it leaves the mailbox, the unread count
drops, and the row is still in the mail table with deleted_at set.
Parity test: ParitySeeded.Mail (DeleteMail).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The page held one-off repairs from Nexus Dashboard (approve known pet
names, find/delete orphaned pet names, remove every buff, fix property
clone IDs, list mission rewards without a commendation price). Removed
the page, its sidebar entry, its /api/maintenance/* routes, the
`maintenance` permission and the two database calls only it used
(GetAllPetNames, FixPropertyCloneIds). The scheduled tasks (lift expired
bans, fill in pet owners, approve known pet names) and property model
import/removal stay.
Check: the Admin sidebar group has no Maintenance link; /maintenance is
a 404; the Tasks page still lists "Approve known pet names" and "Fill in
pet owners"; property import still works; the Permissions page no
longer lists "Data maintenance".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- IUgc::GetUgcProcessTotals sums what the UGC server made, for models and
for car and rocket builds: count, time spent (and how many are timed),
CPU time, average and slowest, average and most memory (estimate),
bricks, triangles and triangles saved (models whose count before hidden
face removal is known). /api/ugc returns them as totals and the UGC page
shows a card for each kind. Parity tested.
- Player models no longer have a shared icon preset: every one is a
different size and shape, so its icon is fitted to it from the settings.
The UGC server ignores a kind:model preset, the dashboard refuses to save
one, and the icon editor hides the type buttons for models. One model's
own icon values still work. Car and rocket build types keep theirs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- resources/*.ini list the 73 settings the catalog knew but the files left
out (dashboard AI helper, backups, metrics, public status, strikes,
property rent and reputation, chat log, contraband, logins...), with
their title, description and default. The test
ShippedFilesListEveryCatalogSetting keeps it that way; with
DLU_WRITE_INI_TEMPLATES=1 it writes the missing ones.
- ConfigSync forgets a setting row when its key has left a file the
server read: from the file, no value set on the dashboard, not a
permission level. Before, rows of removed keys stayed forever.
- Docs: where setting rows come from and when they go, the templates.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/reprocessproperty and the dashboard's Reprocess all models mark the
property's models as priority (ugc.priority, migrations mysql 94 and sqlite
77). The UGC server takes priority models first, polls them even when its
queue is full, and puts them at its front, as it does cars and rockets. The
flag clears once a model is made.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NULL is the owner (every model placed before this, and dashboard imports).
Read by GetPropertyModels and GetModel, written by InsertNewPropertyModel on
MySQL and SQLite, with a parity test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The property page gets a Reprocess all models button (ugc_manage): every
model placed on the property goes back to the UGC server's queue, made
again with the current settings (/api/ugc/reprocess with {property}).
Import models gets Remove all models (properties_import): deletes every
model placed on the property after the property id is typed in, refused
while the property is loaded in a world, as import is.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UGC server stores a model's most detailed level's triangles before it
removed the faces that can't be seen (ugc.triangle_count_before;
migrations sqlite 75, mysql 92), from stats.json when it makes a model
and, once, for the models made before. The UGC page's models list has a
Saved column (the share and number of triangles removed, before/after in
its tooltip), sortable by the share (sort=savings), and the gallery can
sort by most triangles saved.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The models list's Took, CPU and RAM (est.) are separate columns, each
sortable (sort=slowest|cpu|memory). The File column shows the name a
player gave the model where it is placed, with the upload's extension
(the upload's file name is its tooltip), and the name sort uses it
(case-insensitive, so SQLite and MySQL agree).
The config layer test no longer assumes the build's sharedconfig.ini has
no mysql_host; it checks that the database-supplied value isn't used.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The models list has separate Made (when) and Took (the time, CPU and
estimated memory) columns, each sortable in either direction; /api/ugc
takes sort=made (processed_at) besides sort=slowest, and the gallery can
sort by recently made.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UGC server records, with each successful make's time, the worker
thread's CPU time for it and the memory it estimated the job needs (the
figure its memory budget counts; not a measurement)
(process_cpu_ms and process_memory_kb on ugc and ugc_modular_build;
migrations sqlite 74, mysql 91). The UGC page shows them with the time
and duration ("took 12.4 s, CPU 11.9 s, ~96 MB RAM (est.)"), and
durations use the largest units that fit, up to days.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UGC server records how long each successful make took (process_ms on
ugc and ugc_modular_build; migrations sqlite 73, mysql 90). The UGC page
shows it with the time it was made: in the models list's Made column
(sortable, slowest first), in the tiles' tooltips, in the item preview
and in an assembly's References. The gallery can sort by slowest to
make too. Makes from before this are shown without a duration.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cars and rockets built before builds were recorded have no subkey and no
ugc_modular_build row, so the client has no blueprint id to ask for their
icon with. When a character loads, such an item (a ModularBuildComponent
createdLOT with assemblyPartLOTs but no subkey) gets what a new build
gets: a persistent id as its subkey and a ugc_modular_build row with its
modules and owner. The next save keeps the subkey; nothing is dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With UGCUSE3DSERVICES=7:0 (the client's default) the client asks its world for
a blueprint file's MD5 and size (REQUEST_UGC_MANIFEST_INFO, world 27) and then
downloads BrickModels/UserMade/<id % 1000>/<id>.<ext>.sd0. Layouts checked in
the 1.10.64 client: the request is a u64 blueprint id and a u8 resource type;
the answer (UGC_MANIFEST_RESPONSE, client 60) repeats them and adds a u8 valid,
the u32 size and the 16 byte MD5 of the inflated file, 37 bytes after the 0x53
exactly or the client drops it.
- dNet: WorldPackets::RequestUgcManifestInfo, ClientPackets::UgcManifestResponse
(eUgcResourceType), with byte tests against the client's layouts.
- Database: ugc_file_checksums (per model or module combination and file) and
ugc_modular_build.combination_id (migrations 89 / 72), GetUgcFileChecksum
looks a blueprint up as a model, else as a build through its combination.
- UGC server: every download is also written as .sd0 (Sd0::Compress); workers
hand the checksums back and the main thread stores them; old items get their
sd0 icon and checksum, and builds their combination id, once at start-up, a
few per tick; serves <dir>/BrickModels/UserMade/<bucket>/<id>.<ext>.sd0 under
client_path, /<folder>/UserBrickModels and the root (.hkx 404).
- World: UgcManifest answers on the main thread with one indexed query per
request; files not made yet are answered when they are (looked at again
every 5 seconds), and a model in its quiet period is made right away. No
worker threads, HTTP or file reads in the world.
Off by default (ugc_manifest=0): checked in game, the client then downloads
from http://127.0.0.1:80/lwoclient/UserBrickModels/ whatever its boot.cfg says
and logs the player out when it can't connect, so icons need the UGC server on
port 80 of each player's machine. docs/UgcServer.md has the details.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UGC page's List view (models and car/rocket assemblies) and an
assembly's References panel are server-side DataTables like the rest of
the dashboard: sort by clicking a column heading, in either direction,
with the page length and sort remembered per user. They still read
/api/ugc and /api/ugc/assembly/builds, so the prefix search and filters
above the list keep working; the gallery keeps its own pager and sort.
/api/ugc takes reverse=1 for a sort's other direction (the model list's
SQL order and the assembly sort both flip), and the references endpoint
sorts by id, owner, account or state. Tests cover the reversed orders.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
IUgcLookup::ListUgc pages either kind with the UGC search's matching, a state
and a sort, and counts the matches (MySQL and SQLite share the SQL). The UGC
server records each model's bricks and triangles (new ugc columns) so models
can be sorted by size. Cars and rockets are listed as assemblies, one per
combination of modules, with their build type, module names and how many
builds use them; filters for type, state, module and owner. Each assembly's
builds (with where they are) are paged, and a build's assembly can be looked
up for links.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A model with no bricks has nothing to make: it gets its own state
(is_optimized = 3), isn't counted as a failure or retried, shows as Empty
on the UGC page (with its own filter and count), in the status and in
Prometheus, and its downloads answer 404 like HKX. State names come from
the enum (magic_enum). Migrations dlu/mysql/87 and dlu/sqlite/70 move the
rows that failed only because they had no bricks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brighter icons: a world light, a fill from the camera, a highlight, exposure
and contrast; with the defaults the icons' mean luminance matches the game's
own model icons (118 against 120 on a scratch set). Every icon parameter is
listed once (UgcIconParams: key, setting, range, default); the settings,
the dashboard's settings entries and the icon editor come from that list.
Presets per kind (player models, each car or rocket build type from
ModularBuildComponent) and overrides per model or module combination are in
ugc_icon_settings.
Saved models wait ugc_debounce_seconds (sharedconfig) after the owner's last
save before they're made (ugc.process_after); a client asking for one, the
owner leaving the world or a reset ends the wait.
Cars and rockets: one icon per combination of modules (sorted LOTs), shared
by every build of it; builds of a combination made already are marked made
right away, the client's per-blueprint downloads serve the shared files.
The dashboard can delete one item's files, purge by filter or all, preview
icons with any values on the UGC server (/admin routes, master password),
save presets and overrides, and draw a kind's icons again (icons only).
Migrations dlu/mysql/86 and dlu/sqlite/69. Not done yet: the dashboard
editor's lighting controls in the page script (routes are there), docs for
it, the empty-model state, /ugc?item= links, the shared fetch helper; the
storage size and property loading bugs are next.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
IUgcLookup: search ugc and ugc_modular_build by id, creator, property, model
name or LOT, and find where a creation is placed or mailed. The SQL is shared
by MySQL and SQLite (UgcLookupSql.h).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The /ugc page no longer needs the browser to reach the UGC server: the
dashboard fetches its status (/api/ugc/server/status), the files it made
(/api/ugc/files/<kind>/<id>/<file>) and its NIFs converted for the 3D view
(/api/ugc/mesh/<id>, NifFile like the scenery) from ugc_internal_url
(default http://127.0.0.1:2008) with libcurl on the worker threads, keeping
small answers briefly. ugc_public_url is only an "open on the UGC server"
link now.
The page gets an icon gallery beside the list, filtered by kind, state and a
search by id or owner (GetUgcProcessList/GetModularBuildProcessList take a
search), and a viewer: the generated NIF in 3D at any LOD, now or before it
was made again, with wireframe, vertex color and baked lighting switches, the
LXFML beside it, the icon now and before, and the stats with triangles before
and after hidden faces were removed. The status box shows CPU, memory, the
jobs' memory estimate with their limits, and throttling. The settings page
lists the UGC server's new settings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Online state with the time it came up, down/up alerts, a UGC column in the
health history, /api/servers (every server with its process memory and CPU)
and /api/servers/ugc, and UGC gauges for Prometheus.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Deleting an item now follows its ItemComponent delResIndex row in the
DeletionRestrictions table, the way the client's shared inventory code
decides it (LWOInventoryComponent_Common::CanRemoveFromInventory @
0x00ce0d20, CheckDeletionRestrictionIndex @ 0x00c94c20):
- missing, unrestricted, unknown-type or empty rows allow it;
- LOTS_INCLUDED: another item of any listed LOT must remain;
- LOTS_EXCLUDED: other items of every listed LOT must remain;
- ANY_RESTRICTION / ALL_RESTRICTIONS: any / all listed rows allow it;
- ZONE: only in the listed maps; ALWAYS_RESTRICTED: never.
Operators (GM level 9) may delete anything, as in the client. A refused
delete is logged and the item stays.
ItemComponent.minNumRequired is not used: the client never reads it, so
its meaning can't be verified.
Issue 960: the rocket (6416, row 8) and the classic rocket parts (rows 1-3)
have rows that keep at least one rocket or part, so the last rocket can
no longer be deleted and strand the player.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hashed API keys with scope, restrictions, limits, expiry and batched
usage counters, for MySQL and SQLite, with test stubs and parity tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One row per server and minute (packets, bytes, resends, HTTP requests, errors,
bytes and latency percentiles), written in batches and read summed into
buckets for the longer chart ranges.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The XML upload is the one place hand-written XML reaches the game, whose
load path trusts the XML because the server writes it itself. Instead of
making the game skip bad data at load, the upload is checked against
what the load path assumes and refused (400, with the problems) when the
game couldn't load it: required elements, attributes that must parse
(flags read with std::stoul/stoull), required item and mission fields,
known inventory types, mission states and character versions, items and
missions that exist in the CDClient, unique item IDs and slots, and the
acct attribute matching the owner.
Suspicious but loadable content is returned as warnings that need
confirm=true (409 otherwise): contraband (same matching as the world,
now shared in ContrabandRules.h), stacks above the stack size, coins,
level or u-score out of reach, a GM level above the account's.
Contraband marked flag-and-remove is removed only if the uploader asks;
once stored, findings are flagged (CONTRABAND) and audited. The XML
editor shows the findings and offers "Save anyway". Related: issue 1332.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ModularBuildFinish hardcoded the item a finished build becomes (6416 for 3
parts, 8092 for 7) and the car chassis part (8129) that the every-part-
swapped check skips. They now come from ModularBuildComponent: createdLOT,
<numberOfParts> and the <ExamplePartLOT> of the <rootPart> module (new
CDModularBuildComponentTable). Same results with the 1.10.64 cdclient;
tests cover the xml parsing and the lookup.
Refs #691
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The client sends SetBBBAutosave (996) with the model being built every five
minutes, before an AFK kick and before quitting, and expects the server to
rebuild an unfinished model later (RebuildBBBAutosaveMsg). This keeps the
last one per character with the model items that were in the BBB inventory
at the time. MySQL 82 / SQLite 65; parity test included.
Refs #1632
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new server (dUgcServer, started by master with enable_ugc_server=1) that
takes unprocessed ugc and ugc_modular_build rows from the database and makes
what the client downloads with UGCUSE3DSERVICES: an optimized NIF (hidden
faces removed, ambient occlusion baked into vertex colors) and a 128px DDS
icon for player models, rendered by a software rasterizer from the client's
LDD brick primitives, and icons for cars and rockets assembled from their
modules per ModularBuildComponent/ModuleComponent. It serves them, with the
models' LXFML, over HTTP in the client's UGCC<dc>/3DOPTIMIZED and
IMAGE128DDS layout with .gz and .checksum files, and keeps its folder under
a size cap.
Processing state lives in ugc.is_optimized plus new processed_at,
process_attempts and process_error columns (and the same on
ugc_modular_build). ServiceType::UGC is appended. NifFile moves to dCommon
and records named node transforms for the modules' attach points.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pet_names gets a pet_lot column (mysql 80, sqlite 63). The world writes
it whenever it saves a pet name, from the pet entity's LOT, and fills it
in for older rows when the owner loads into a world (from the pets the
game loads for that character, only where it is still missing).
The dashboard's pet name tables read pet_lot instead of scanning the
owner's character XML; pets without it yet show as Unknown.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Property worlds now give their property reputation for the time other people spend on it, which
fills the property lists and the news screen's Today's Top Properties. Visitors are counted per
account; the owner's account, accounts linked to it and staff don't count. A visit earns nothing
for the first property_reputation_min_visit seconds (WorldConfig's propertyReputationDelay), then
each active minute (the visitor moved) earns reputationPerMinute times a multiplier, for a capped
number of minutes per visit. Repeat visitors earn less the more recent days they already gave
reputation, and each visitor and each property have a daily cap. Every parameter is a setting;
what each account gave each property per day is kept in property_reputation_visits. The rules are
pure functions with unit tests.
Fixes#636Fixes#637
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
