The page held one-off repairs from Nexus Dashboard (approve known pet
names, find/delete orphaned pet names, remove every buff, fix property
clone IDs, list mission rewards without a commendation price). Removed
the page, its sidebar entry, its /api/maintenance/* routes, the
`maintenance` permission and the two database calls only it used
(GetAllPetNames, FixPropertyCloneIds). The scheduled tasks (lift expired
bans, fill in pet owners, approve known pet names) and property model
import/removal stay.
Check: the Admin sidebar group has no Maintenance link; /maintenance is
a 404; the Tasks page still lists "Approve known pet names" and "Fill in
pet owners"; property import still works; the Permissions page no
longer lists "Data maintenance".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- IUgc::GetUgcProcessTotals sums what the UGC server made, for models and
for car and rocket builds: count, time spent (and how many are timed),
CPU time, average and slowest, average and most memory (estimate),
bricks, triangles and triangles saved (models whose count before hidden
face removal is known). /api/ugc returns them as totals and the UGC page
shows a card for each kind. Parity tested.
- Player models no longer have a shared icon preset: every one is a
different size and shape, so its icon is fitted to it from the settings.
The UGC server ignores a kind:model preset, the dashboard refuses to save
one, and the icon editor hides the type buttons for models. One model's
own icon values still work. Car and rocket build types keep theirs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- resources/*.ini list the 73 settings the catalog knew but the files left
out (dashboard AI helper, backups, metrics, public status, strikes,
property rent and reputation, chat log, contraband, logins...), with
their title, description and default. The test
ShippedFilesListEveryCatalogSetting keeps it that way; with
DLU_WRITE_INI_TEMPLATES=1 it writes the missing ones.
- ConfigSync forgets a setting row when its key has left a file the
server read: from the file, no value set on the dashboard, not a
permission level. Before, rows of removed keys stayed forever.
- Docs: where setting rows come from and when they go, the templates.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/reprocessproperty and the dashboard's Reprocess all models mark the
property's models as priority (ugc.priority, migrations mysql 94 and sqlite
77). The UGC server takes priority models first, polls them even when its
queue is full, and puts them at its front, as it does cars and rockets. The
flag clears once a model is made.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NULL is the owner (every model placed before this, and dashboard imports).
Read by GetPropertyModels and GetModel, written by InsertNewPropertyModel on
MySQL and SQLite, with a parity test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The property page gets a Reprocess all models button (ugc_manage): every
model placed on the property goes back to the UGC server's queue, made
again with the current settings (/api/ugc/reprocess with {property}).
Import models gets Remove all models (properties_import): deletes every
model placed on the property after the property id is typed in, refused
while the property is loaded in a world, as import is.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UGC server stores a model's most detailed level's triangles before it
removed the faces that can't be seen (ugc.triangle_count_before;
migrations sqlite 75, mysql 92), from stats.json when it makes a model
and, once, for the models made before. The UGC page's models list has a
Saved column (the share and number of triangles removed, before/after in
its tooltip), sortable by the share (sort=savings), and the gallery can
sort by most triangles saved.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The models list's Took, CPU and RAM (est.) are separate columns, each
sortable (sort=slowest|cpu|memory). The File column shows the name a
player gave the model where it is placed, with the upload's extension
(the upload's file name is its tooltip), and the name sort uses it
(case-insensitive, so SQLite and MySQL agree).
The config layer test no longer assumes the build's sharedconfig.ini has
no mysql_host; it checks that the database-supplied value isn't used.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The models list has separate Made (when) and Took (the time, CPU and
estimated memory) columns, each sortable in either direction; /api/ugc
takes sort=made (processed_at) besides sort=slowest, and the gallery can
sort by recently made.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UGC server records, with each successful make's time, the worker
thread's CPU time for it and the memory it estimated the job needs (the
figure its memory budget counts; not a measurement)
(process_cpu_ms and process_memory_kb on ugc and ugc_modular_build;
migrations sqlite 74, mysql 91). The UGC page shows them with the time
and duration ("took 12.4 s, CPU 11.9 s, ~96 MB RAM (est.)"), and
durations use the largest units that fit, up to days.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UGC server records how long each successful make took (process_ms on
ugc and ugc_modular_build; migrations sqlite 73, mysql 90). The UGC page
shows it with the time it was made: in the models list's Made column
(sortable, slowest first), in the tiles' tooltips, in the item preview
and in an assembly's References. The gallery can sort by slowest to
make too. Makes from before this are shown without a duration.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cars and rockets built before builds were recorded have no subkey and no
ugc_modular_build row, so the client has no blueprint id to ask for their
icon with. When a character loads, such an item (a ModularBuildComponent
createdLOT with assemblyPartLOTs but no subkey) gets what a new build
gets: a persistent id as its subkey and a ugc_modular_build row with its
modules and owner. The next save keeps the subkey; nothing is dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With UGCUSE3DSERVICES=7:0 (the client's default) the client asks its world for
a blueprint file's MD5 and size (REQUEST_UGC_MANIFEST_INFO, world 27) and then
downloads BrickModels/UserMade/<id % 1000>/<id>.<ext>.sd0. Layouts checked in
the 1.10.64 client: the request is a u64 blueprint id and a u8 resource type;
the answer (UGC_MANIFEST_RESPONSE, client 60) repeats them and adds a u8 valid,
the u32 size and the 16 byte MD5 of the inflated file, 37 bytes after the 0x53
exactly or the client drops it.
- dNet: WorldPackets::RequestUgcManifestInfo, ClientPackets::UgcManifestResponse
(eUgcResourceType), with byte tests against the client's layouts.
- Database: ugc_file_checksums (per model or module combination and file) and
ugc_modular_build.combination_id (migrations 89 / 72), GetUgcFileChecksum
looks a blueprint up as a model, else as a build through its combination.
- UGC server: every download is also written as .sd0 (Sd0::Compress); workers
hand the checksums back and the main thread stores them; old items get their
sd0 icon and checksum, and builds their combination id, once at start-up, a
few per tick; serves <dir>/BrickModels/UserMade/<bucket>/<id>.<ext>.sd0 under
client_path, /<folder>/UserBrickModels and the root (.hkx 404).
- World: UgcManifest answers on the main thread with one indexed query per
request; files not made yet are answered when they are (looked at again
every 5 seconds), and a model in its quiet period is made right away. No
worker threads, HTTP or file reads in the world.
Off by default (ugc_manifest=0): checked in game, the client then downloads
from http://127.0.0.1:80/lwoclient/UserBrickModels/ whatever its boot.cfg says
and logs the player out when it can't connect, so icons need the UGC server on
port 80 of each player's machine. docs/UgcServer.md has the details.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UGC page's List view (models and car/rocket assemblies) and an
assembly's References panel are server-side DataTables like the rest of
the dashboard: sort by clicking a column heading, in either direction,
with the page length and sort remembered per user. They still read
/api/ugc and /api/ugc/assembly/builds, so the prefix search and filters
above the list keep working; the gallery keeps its own pager and sort.
/api/ugc takes reverse=1 for a sort's other direction (the model list's
SQL order and the assembly sort both flip), and the references endpoint
sorts by id, owner, account or state. Tests cover the reversed orders.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
IUgcLookup::ListUgc pages either kind with the UGC search's matching, a state
and a sort, and counts the matches (MySQL and SQLite share the SQL). The UGC
server records each model's bricks and triangles (new ugc columns) so models
can be sorted by size. Cars and rockets are listed as assemblies, one per
combination of modules, with their build type, module names and how many
builds use them; filters for type, state, module and owner. Each assembly's
builds (with where they are) are paged, and a build's assembly can be looked
up for links.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A model with no bricks has nothing to make: it gets its own state
(is_optimized = 3), isn't counted as a failure or retried, shows as Empty
on the UGC page (with its own filter and count), in the status and in
Prometheus, and its downloads answer 404 like HKX. State names come from
the enum (magic_enum). Migrations dlu/mysql/87 and dlu/sqlite/70 move the
rows that failed only because they had no bricks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brighter icons: a world light, a fill from the camera, a highlight, exposure
and contrast; with the defaults the icons' mean luminance matches the game's
own model icons (118 against 120 on a scratch set). Every icon parameter is
listed once (UgcIconParams: key, setting, range, default); the settings,
the dashboard's settings entries and the icon editor come from that list.
Presets per kind (player models, each car or rocket build type from
ModularBuildComponent) and overrides per model or module combination are in
ugc_icon_settings.
Saved models wait ugc_debounce_seconds (sharedconfig) after the owner's last
save before they're made (ugc.process_after); a client asking for one, the
owner leaving the world or a reset ends the wait.
Cars and rockets: one icon per combination of modules (sorted LOTs), shared
by every build of it; builds of a combination made already are marked made
right away, the client's per-blueprint downloads serve the shared files.
The dashboard can delete one item's files, purge by filter or all, preview
icons with any values on the UGC server (/admin routes, master password),
save presets and overrides, and draw a kind's icons again (icons only).
Migrations dlu/mysql/86 and dlu/sqlite/69. Not done yet: the dashboard
editor's lighting controls in the page script (routes are there), docs for
it, the empty-model state, /ugc?item= links, the shared fetch helper; the
storage size and property loading bugs are next.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
IUgcLookup: search ugc and ugc_modular_build by id, creator, property, model
name or LOT, and find where a creation is placed or mailed. The SQL is shared
by MySQL and SQLite (UgcLookupSql.h).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The /ugc page no longer needs the browser to reach the UGC server: the
dashboard fetches its status (/api/ugc/server/status), the files it made
(/api/ugc/files/<kind>/<id>/<file>) and its NIFs converted for the 3D view
(/api/ugc/mesh/<id>, NifFile like the scenery) from ugc_internal_url
(default http://127.0.0.1:2008) with libcurl on the worker threads, keeping
small answers briefly. ugc_public_url is only an "open on the UGC server"
link now.
The page gets an icon gallery beside the list, filtered by kind, state and a
search by id or owner (GetUgcProcessList/GetModularBuildProcessList take a
search), and a viewer: the generated NIF in 3D at any LOD, now or before it
was made again, with wireframe, vertex color and baked lighting switches, the
LXFML beside it, the icon now and before, and the stats with triangles before
and after hidden faces were removed. The status box shows CPU, memory, the
jobs' memory estimate with their limits, and throttling. The settings page
lists the UGC server's new settings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Online state with the time it came up, down/up alerts, a UGC column in the
health history, /api/servers (every server with its process memory and CPU)
and /api/servers/ugc, and UGC gauges for Prometheus.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Deleting an item now follows its ItemComponent delResIndex row in the
DeletionRestrictions table, the way the client's shared inventory code
decides it (LWOInventoryComponent_Common::CanRemoveFromInventory @
0x00ce0d20, CheckDeletionRestrictionIndex @ 0x00c94c20):
- missing, unrestricted, unknown-type or empty rows allow it;
- LOTS_INCLUDED: another item of any listed LOT must remain;
- LOTS_EXCLUDED: other items of every listed LOT must remain;
- ANY_RESTRICTION / ALL_RESTRICTIONS: any / all listed rows allow it;
- ZONE: only in the listed maps; ALWAYS_RESTRICTED: never.
Operators (GM level 9) may delete anything, as in the client. A refused
delete is logged and the item stays.
ItemComponent.minNumRequired is not used: the client never reads it, so
its meaning can't be verified.
Issue 960: the rocket (6416, row 8) and the classic rocket parts (rows 1-3)
have rows that keep at least one rocket or part, so the last rocket can
no longer be deleted and strand the player.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hashed API keys with scope, restrictions, limits, expiry and batched
usage counters, for MySQL and SQLite, with test stubs and parity tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One row per server and minute (packets, bytes, resends, HTTP requests, errors,
bytes and latency percentiles), written in batches and read summed into
buckets for the longer chart ranges.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The XML upload is the one place hand-written XML reaches the game, whose
load path trusts the XML because the server writes it itself. Instead of
making the game skip bad data at load, the upload is checked against
what the load path assumes and refused (400, with the problems) when the
game couldn't load it: required elements, attributes that must parse
(flags read with std::stoul/stoull), required item and mission fields,
known inventory types, mission states and character versions, items and
missions that exist in the CDClient, unique item IDs and slots, and the
acct attribute matching the owner.
Suspicious but loadable content is returned as warnings that need
confirm=true (409 otherwise): contraband (same matching as the world,
now shared in ContrabandRules.h), stacks above the stack size, coins,
level or u-score out of reach, a GM level above the account's.
Contraband marked flag-and-remove is removed only if the uploader asks;
once stored, findings are flagged (CONTRABAND) and audited. The XML
editor shows the findings and offers "Save anyway". Related: issue 1332.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ModularBuildFinish hardcoded the item a finished build becomes (6416 for 3
parts, 8092 for 7) and the car chassis part (8129) that the every-part-
swapped check skips. They now come from ModularBuildComponent: createdLOT,
<numberOfParts> and the <ExamplePartLOT> of the <rootPart> module (new
CDModularBuildComponentTable). Same results with the 1.10.64 cdclient;
tests cover the xml parsing and the lookup.
Refs #691
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The client sends SetBBBAutosave (996) with the model being built every five
minutes, before an AFK kick and before quitting, and expects the server to
rebuild an unfinished model later (RebuildBBBAutosaveMsg). This keeps the
last one per character with the model items that were in the BBB inventory
at the time. MySQL 82 / SQLite 65; parity test included.
Refs #1632
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new server (dUgcServer, started by master with enable_ugc_server=1) that
takes unprocessed ugc and ugc_modular_build rows from the database and makes
what the client downloads with UGCUSE3DSERVICES: an optimized NIF (hidden
faces removed, ambient occlusion baked into vertex colors) and a 128px DDS
icon for player models, rendered by a software rasterizer from the client's
LDD brick primitives, and icons for cars and rockets assembled from their
modules per ModularBuildComponent/ModuleComponent. It serves them, with the
models' LXFML, over HTTP in the client's UGCC<dc>/3DOPTIMIZED and
IMAGE128DDS layout with .gz and .checksum files, and keeps its folder under
a size cap.
Processing state lives in ugc.is_optimized plus new processed_at,
process_attempts and process_error columns (and the same on
ugc_modular_build). ServiceType::UGC is appended. NifFile moves to dCommon
and records named node transforms for the modules' attach points.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pet_names gets a pet_lot column (mysql 80, sqlite 63). The world writes
it whenever it saves a pet name, from the pet entity's LOT, and fills it
in for older rows when the owner loads into a world (from the pets the
game loads for that character, only where it is still missing).
The dashboard's pet name tables read pet_lot instead of scanning the
owner's character XML; pets without it yet show as Unknown.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Property worlds now give their property reputation for the time other people spend on it, which
fills the property lists and the news screen's Today's Top Properties. Visitors are counted per
account; the owner's account, accounts linked to it and staff don't count. A visit earns nothing
for the first property_reputation_min_visit seconds (WorldConfig's propertyReputationDelay), then
each active minute (the visitor moved) earns reputationPerMinute times a multiplier, for a capped
number of minutes per visit. Repeat visitors earn less the more recent days they already gave
reputation, and each visitor and each property have a daily cap. Every parameter is a setting;
what each account gave each property per day is kept in property_reputation_visits. The rules are
pure functions with unit tests.
Fixes#636Fixes#637
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Off unless property_rent_enabled is on. Each property world's rent comes from its PropertyTemplate
row (minimumPrice every rentDuration x durationType; Block Yard is free), unless the new Property
Rent dashboard page sets another price or period (property_rent_manage). Rent is taken from the
owner's coins shortly after their character loads, with a mail receipt; unpaid rent is mailed and,
after property_rent_grace_days, makes the property private until it is paid, like live. The
property management component refuses public or best friends privacy while rent is overdue and a
property world that loads overdue makes itself private. Property game messages are unchanged.
Fixes#943
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Staff list contraband items on a new dashboard page (item search, reason, flag or flag and remove;
contraband_manage to edit, reports_view to see). World servers check every inventory when a
character loads and every item a player receives: each find is an economy flag of the new kind
Contraband, shown with the other flags and in the character's related data. Items marked for
removal are taken away, with a character snapshot kept first so they can be given back, an audit
entry and a mail or chat message telling the player why. Staff are skipped unless
contraband_ignore_staff is off. Worlds reload the list when it changes (RELOAD_CONTRABAND, added
at the end of ePlayerAction).
Fixes#1563
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every character gets a save generation in charxml. A world bumps it when it loads the character for
play, and every save from that world only goes through while the stored generation is still the
one it loaded or last saved (and moves it on). Dashboard edits, restores and maintenance writes bump
it too. A world that lost the character to another world (a disconnect noticed late, a zone
transfer, an instance migration) or to a dashboard edit can no longer overwrite the newer data: the
save is refused, logged and audited as stale_save_refused, the world stops saving that character
and a player still connected to it is disconnected with the save failure reason so they reload.
Fixes#639
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Leave staff out of the currency and U-score reports by excluding the few staff characters,
instead of joining every ledger row to its character and account.
- Top earners: total per character first and look up names for the top rows only.
- The places list and the activity report read map events for every kind in one query
(GetMapZonesAllKinds) instead of one query per kind.
With about 1M currency rows and 650k map event rows (90 days) on MariaDB: currency 1.8 s to
0.8 s, top earners 2.4 s to 0.75 s, places 2.6 s to 0.5 s, activity 2.6 s to 1.7 s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Read activity and map_id as signed: a row with map_id -1 made the page fail with a server error
when sorted by map.
- A search counts and pages through the characters it matches (index on character_id) instead
of joining every row of the log to a name; when the matches are a large part of the log the
page still walks the log in order.
- Sorted by character name, only the row ids are sorted and the page's rows are read after.
On a log of about 800k rows this takes searches with few matches from about 1 s to 0.1 s and
name sorts from about 3 s to 0.5 s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The NexusDashboard-parity dashboard (dDashboardServer) and everything built on it on the experimental branch:
accounts, characters, properties and moderation tools, permissions shared with in-game slash commands, economy
reports, World 3D and property 3D views with client scenery, scheduled events (features, vanity changes, live
events, announcements, restarts), vanity files and events, the CDClient browser, the message inspector with saved
captures, chat filter tools, community challenges, live ops, the AI moderator helper, and the server-side changes
they need.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat: enemies now use weights on their attacks
tested that 8 times out of 10, in close range, spiders did a web attack instead of a melee attack, vs the prior behavior of always following a pattern
fixes#2002
* feedback
* fix: security vulnerabilities
Tested that all functions related to the touched files work
will test sqlite on a CI build
* fix failing test
* ai feedback
* add buffer size checking
* use c_str
* dont log session key
* Try this for a mac definition
* be quiet apple
* feat: re-write persistent object ID tracker
Features:
- Remove random objectIDs entirely
- Replace random objectIDs with persistentIDs
- Remove the need to contact the MASTER server for a persistent ID
- Add persistent ID logic to WorldServers that use transactions to guarantee unique IDs no matter when they are generated
- Default character xml version to be the most recent one
Fixes:
- Return optional from GetModel (and check for nullopt where it may exist)
- Regenerate inventory item ids on first login to be unique item IDs (fixes all those random IDs
Pet IDs and subkeys are left alone and are assumed to be reserved (checks are there to prevent this)
There is also duplicate check logic in place for properties and UGC/Models
* Update comment and log
* fix: sqlite transaction bug
* fix colliding temp item ids
temp items should not be saved. would cause issues between worlds as experienced before this commit
* feat: Remove PERSISTENT ObjectID bit because it's not an ObjectID bit
TODO: Need to add character save migration for the pet subkey in the inventory
Tested that the migrations work on mysql and sqlite and that properties have all their contents as before.
Need to test pets still
* fix: ugc, pet ids. remove persistent bit
* feat: convert character ids to 64 bits
remove all usages of the PERSISTENT bit with regards to storing of playerIDs on the server. the bit does not exist and was a phantom in the first place.
Tested that a full playthrough of ag, ns and gf was still doable. slash commands work, ugc works, friends works, ignore list works, properties work and have names, teaming works.
migrating an old mysql database works . need to test an old sqlite database
* fix sqlite migration
* remove nd specific column migration
Tested that models are migrated to the new format a-ok
Tested that the new logic works as expected.
Old code needs to be kept so that models in both states can be brought to modern standards
* Break out changes into a smaller subset
* NL@EOF
* fix windows bs
add player ws updates
add websocket docs
* tested everything to make sure it works
* Address Feedback
