Glom models (multishader) drew their trees, rocks, fences and water white.
Their conversions kept on disk from before meshes carried their multishader
tag were still served (the cache format version was never bumped), so every
part fell back to the LEGO shader, whose texture alpha lays the (mostly grey)
texture over the vertex colors that hold the actual colors. Browsers also
kept those models for a week.
- Bump the conversion format so old conversions are made again, and put it
in the manifests; the viewers add it to model and texture URLs.
- Light scenery as the client's shaders do: the scene's sun and ambient
light from its .lvl (read as level_read_lighting_info, 0x0102f8f0, and
EnvironmentManager::SetLightEnv, 0x01088aa0, do), per vertex, clamped,
instead of the view's own lights, environment map and tone mapping. The
zone takes the lighting most of its objects' scenes have.
- Programmable shaders read vertex colors and ignore NiMaterialProperty's
color and alpha; unlit and untextured shaders (by mapShaders gameValue)
leave out lighting or the texture. Fixed function stays as it was.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UGC server stores a model's most detailed level's triangles before it
removed the faces that can't be seen (ugc.triangle_count_before;
migrations sqlite 75, mysql 92), from stats.json when it makes a model
and, once, for the models made before. The UGC page's models list has a
Saved column (the share and number of triangles removed, before/after in
its tooltip), sortable by the share (sort=savings), and the gallery can
sort by most triangles saved.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The models list's Took, CPU and RAM (est.) are separate columns, each
sortable (sort=slowest|cpu|memory). The File column shows the name a
player gave the model where it is placed, with the upload's extension
(the upload's file name is its tooltip), and the name sort uses it
(case-insensitive, so SQLite and MySQL agree).
The config layer test no longer assumes the build's sharedconfig.ini has
no mysql_host; it checks that the database-supplied value isn't used.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The models list has separate Made (when) and Took (the time, CPU and
estimated memory) columns, each sortable in either direction; /api/ugc
takes sort=made (processed_at) besides sort=slowest, and the gallery can
sort by recently made.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UGC server records, with each successful make's time, the worker
thread's CPU time for it and the memory it estimated the job needs (the
figure its memory budget counts; not a measurement)
(process_cpu_ms and process_memory_kb on ugc and ugc_modular_build;
migrations sqlite 74, mysql 91). The UGC page shows them with the time
and duration ("took 12.4 s, CPU 11.9 s, ~96 MB RAM (est.)"), and
durations use the largest units that fit, up to days.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UGC server records how long each successful make took (process_ms on
ugc and ugc_modular_build; migrations sqlite 73, mysql 90). The UGC page
shows it with the time it was made: in the models list's Made column
(sortable, slowest first), in the tiles' tooltips, in the item preview
and in an assembly's References. The gallery can sort by slowest to
make too. Makes from before this are shown without a duration.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dashboard shows worlds master has launched but that aren't connected
yet (Starting) and those shutting down, on the main page's world list
and in /api/worlds (state: starting|stopping), without a shutdown
button. Master's server list now carries each world's state (appended
after the UGC fields, one byte per world) and master pushes the list to
the dashboard whenever a world is launched, becomes ready, is told to
shut down or goes away, instead of the dashboard only seeing it on its
30 second poll. Starting worlds are kept apart from the running ones,
so counts, events and shutdown requests still only see running worlds.
prestart_worlds (masterconfig.ini, zone ids) lists the worlds master
starts when prestart_servers is on, instead of the hardcoded character
select (0) and Venture Explorer (1000), which stay the default when it
is missing or empty. It is on the Settings page as a zone list (restart
only), shown when prestart_servers is on.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UGC page's List view (models and car/rocket assemblies) and an
assembly's References panel are server-side DataTables like the rest of
the dashboard: sort by clicking a column heading, in either direction,
with the page length and sort remembered per user. They still read
/api/ugc and /api/ugc/assembly/builds, so the prefix search and filters
above the list keep working; the gallery keeps its own pager and sort.
/api/ugc takes reverse=1 for a sort's other direction (the model list's
SQL order and the assembly sort both flip), and the references endpoint
sorts by id, owner, account or state. Tests cover the reversed orders.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The settings catalog gives the UGC server a category of its own (serving,
processing, models, storage, icons). The UGC page shows those sections next to
the server status, the purge tools and the icon presets, read from the catalog
and saved through the Settings page's own path; each links to its entry on the
Settings page, which now opens a #file/name link at that setting.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brighter icons: a world light, a fill from the camera, a highlight, exposure
and contrast; with the defaults the icons' mean luminance matches the game's
own model icons (118 against 120 on a scratch set). Every icon parameter is
listed once (UgcIconParams: key, setting, range, default); the settings,
the dashboard's settings entries and the icon editor come from that list.
Presets per kind (player models, each car or rocket build type from
ModularBuildComponent) and overrides per model or module combination are in
ugc_icon_settings.
Saved models wait ugc_debounce_seconds (sharedconfig) after the owner's last
save before they're made (ugc.process_after); a client asking for one, the
owner leaving the world or a reset ends the wait.
Cars and rockets: one icon per combination of modules (sorted LOTs), shared
by every build of it; builds of a combination made already are marked made
right away, the client's per-blueprint downloads serve the shared files.
The dashboard can delete one item's files, purge by filter or all, preview
icons with any values on the UGC server (/admin routes, master password),
save presets and overrides, and draw a kind's icons again (icons only).
Migrations dlu/mysql/86 and dlu/sqlite/69. Not done yet: the dashboard
editor's lighting controls in the page script (routes are there), docs for
it, the empty-model state, /ugc?item= links, the shared fetch helper; the
storage size and property loading bugs are next.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
UGC Search (/ugc_search) finds creations by name, id or LOT with where each
one is. Property pages and character inventories show the icon the UGC server
made of each creation, its state and a link to /ugc?item=&kind=, for whoever
may view the page.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Online state with the time it came up, down/up alerts, a UGC column in the
health history, /api/servers (every server with its process memory and CPU)
and /api/servers/ugc, and UGC gauges for Prometheus.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bearer keys (dlk_...) are checked per request against their owner's
current account (ban, lock, demotion, sign out everywhere stop or narrow
them at once) and their scope: permission routes need the permission in
scope, read-only keys only read, level-only routes need an all-permission
key, and session-only paths (sign-in, password, 2FA, key management)
are never reachable with a key. Per-key rate limit and daily quota with
429 and X-RateLimit/X-Quota/Retry-After headers; usage is written in
batches every minute. WebSocket subscriptions honour the scope too.
Routes to list, make, rotate and revoke keys; staff with
api_keys_manage can see and revoke others' keys under the rank rules.
POST /api/auth/token now makes an all-permission key. Audit entries for
create/rotate/revoke/denied, and actions done with a key are attributed
to "user (key name)".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keeps every server's traffic reports (last hour at one second, per-minute
rows to server_traffic once a minute, pruned after traffic_days) and shows
packets, bytes, HTTP requests and latency per second, per server, with the
busiest message types and HTTP routes. Live over the traffic WebSocket topic;
1 hour, 24 hours and 7 days ranges. The same counters are in /metrics.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The XML upload is the one place hand-written XML reaches the game, whose
load path trusts the XML because the server writes it itself. Instead of
making the game skip bad data at load, the upload is checked against
what the load path assumes and refused (400, with the problems) when the
game couldn't load it: required elements, attributes that must parse
(flags read with std::stoul/stoull), required item and mission fields,
known inventory types, mission states and character versions, items and
missions that exist in the CDClient, unique item IDs and slots, and the
acct attribute matching the owner.
Suspicious but loadable content is returned as warnings that need
confirm=true (409 otherwise): contraband (same matching as the world,
now shared in ContrabandRules.h), stacks above the stack size, coins,
level or u-score out of reach, a GM level above the account's.
Contraband marked flag-and-remove is removed only if the uploader asks;
once stored, findings are flagged (CONTRABAND) and audited. The XML
editor shows the findings and offers "Save anyway". Related: issue 1332.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new server (dUgcServer, started by master with enable_ugc_server=1) that
takes unprocessed ugc and ugc_modular_build rows from the database and makes
what the client downloads with UGCUSE3DSERVICES: an optimized NIF (hidden
faces removed, ambient occlusion baked into vertex colors) and a 128px DDS
icon for player models, rendered by a software rasterizer from the client's
LDD brick primitives, and icons for cars and rockets assembled from their
modules per ModularBuildComponent/ModuleComponent. It serves them, with the
models' LXFML, over HTTP in the client's UGCC<dc>/3DOPTIMIZED and
IMAGE128DDS layout with .gz and .checksum files, and keeps its folder under
a size cap.
Processing state lives in ugc.is_optimized plus new processed_at,
process_attempts and process_error columns (and the same on
ugc_modular_build). ServiceType::UGC is appended. NifFile moves to dCommon
and records named node transforms for the modules' attach points.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The viewer treated every blended texture's alpha as opacity, so models
whose shader uses the alpha for something else rendered see-through. The
scenery manifest now carries each model's shader (RenderComponent.
shader_id via mapShaders) and the multishader tag table; meshes carry
their S##__ tag, read the way LWOBaseRenderComponent::AddObjectToRenderPipe
does (S%d, else _S%d, outside 3..108 the LEGO shader). Per res/shaders:
the LEGO lighting shaders lerp the texture over the vertex colors
(decal), LEGO items and terrain meshes ignore its alpha, everything else
keeps opacity. Pure rules unit tested.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
eReplicaComponentType had the destroyable component's registry type (7) named
BUFF, the real buff component (98) as BUFF_REAL, and a made-up DESTROYABLE =
1000 that DestroyableComponent was stored under. Now DESTROYABLE = 7 and
BUFF = 98, as in ComponentsRegistry and the client.
Undone with it:
- Destructible stats came from whichever of the "buff" (7), quick build and
collectible registry ids was set, so the few objects without a type 7 entry
read a DestructibleComponent row with an unrelated id (the NJ dragon relics
16482-16485 via their collectible id, 125 quick build LOTs when placed with
is_smashable). The type 7 entry is used now; objects without one keep the
defaults (is_smashable objects: 1 health, smashable, factions -1 and 6;
collectibles: an empty destroyable). The client does the same
(LWODestroyableComponent::AllocateComponents / DoObjectComponentLoad).
- DestroyableComponent::Reinitialize, an unused copy of that pick order.
- WriteComponents' destroyableSerialized flags: the components are written from
a list in the client's order, and where the destroyable goes (its own place
after the buff, right before a quick build that has no registry entry for it,
or after the render component) is one function.
- The dashboard's registry 7 -> DESTROYABLE mapping; the destroyable type also
has a name there now (1000 was outside magic_enum's range).
Component types are not stored or sent as enum numbers anywhere besides the
CDClient's own values, which now match. migrations/cdserver/4 is unrelated (it
restores LOT 12916's registry rows that migration 0 overwrote) and stays.
Verified: dGameTests ReplicaComponentOrderTests serialize players, enemies,
smashables, quick builds and collectibles with and without a registry entry,
NPCs, pets, vehicles, models and an entity with every listed component with
the new code and a frozen copy of the old WriteComponents, and expect the same
bits for construction and serialization (a deliberately wrong destroyable place
fails them). Full ctest passes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Opening a zone the first time built its terrain, scene objects and manifests
and ran ImageMagick on the web thread, stalling the dashboard for seconds.
- Workers: the shared pool plus Workers::Reply (answer at once when built,
else from a worker via Web::Defer).
- terrain_chunks/terrain_layers/scene/paths/scenery/flairs (world3d, property
and showcase routes) and terrain textures go through it; results are built
once in OnceCaches, the .raw is read once per zone for chunks, layers and
flairs, deflated bodies are cached thread-safely.
- ImageMagick conversions are deduplicated and written under a temporary name.
- Workers don't query the CDClient, read settings or call mongoose: ZoneTable,
render components, flairs, object names, LOT kinds and terrain texture names
are read at startup; client_location is read once; base64 is plain C++.
- Logger writes one line at a time (mutex; localtime's buffer is shared).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every MASTER service packet is now an LUBitStream struct (docs/PacketArchitecture.md,
PR 14) and the master server's switch is a dispatch map (PacketDispatcher), as are the
master handlers of the world, chat and dashboard servers.
- dNet/MasterPackets.h: RequestZoneTransfer, RequestZoneTransferResponse, ServerInfo,
RequestSessionKey, SetSessionKey, SessionKeyResponse, NewSessionAlert, PlayerAdded /
PlayerRemoved, CreatePrivateZone, RequestPrivateZone (passwords still cut to 50
characters when read), WorldReady, WorldReadyInfo (WORLD_READY to the dashboard),
PrepZone, Shutdown, ShutdownResponse, WorldShutDown (SHUTDOWN_RESPONSE to the
dashboard), ShutdownUniverse, AffirmTransferRequest/Response, RequestServerList,
ServerListResponse, DashboardShutdown, ConfigReload, InstanceShutdown. The Send*
functions are gone; MasterPackets::SendToMaster(msg) and SendTo(sysAddr, msg) send a
struct.
- The dashboard and instance migration structs (PlayerAction, DataChanged, Dashboard
messages, MessageCapture, InstanceMigration) are LUBitStreams of the MASTER service now
and moved to dNet/master/, included by MasterPackets.h. Their payloads are unchanged;
master forwards them by re-serializing the struct instead of copying raw bytes.
- InstanceManager, ZoneInstanceManager, MigrationCoordinator, dServer (server info, zone
transfer response), auth (SET_SESSION_KEY), the world (session keys, player added and
removed, world ready, shutdown response, affirmations, prep zone, shutdown universe) and
the dashboard (server list, instance shutdown, config reload, announcements, player
actions, message capture) send and read structs.
- The login stamps are a `stamps` field of RequestZoneTransfer and
RequestZoneTransferResponse (read leniently as before: a message without them reads as
empty); master adds its stamps in the REQUEST_ZONE_TRANSFER handler and when it answers,
as it did.
- InstanceManager::GetInstanceBySysAddr takes a const address.
Verified: tests/dGameTests/dNetTests/Legacy/MasterPacketsLegacy.h is a verbatim copy of
the old writers and readers; MasterPacketsTests requires identical bytes for a grid of
inputs, checks the old readers read what the structs write, round trips and truncation,
hand written golden packets, that zone transfers without stamps still read, that the dashboard/migration structs write what
"header + Serialize" wrote, and that the dispatcher drops truncated packets. No wire
bytes changed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dashboard's web server answers one request at a time, so converting a big
.nif (glom files up to tens of MB) held up every other request, flairs included.
- dWeb: Web::Defer hands a request to another thread; the reply is sent from the
web thread on its next poll (DeferredQueue). A client that leaves first cancels
it and the late reply is dropped. The synchronous route API is unchanged.
- Web::Shutdown closes connections while the state their close events touch is
still alive; the destructor no longer runs handlers during static destruction
(stopping the dashboard aborted in ~WSClient).
- WorkerPool: priority lanes, with one thread only for urgent work (flairs,
small models, textures), and limited background work.
- Scenery: mesh and texture routes (and the showcase's) convert on the pool;
thread-safe memory and disk caches, one conversion per model at a time with
waiters sharing it; zones are converted ahead onto the disk cache while viewed.
- Setting scenery_workers (0: half the cores, 2 to 4).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mysqldump was given --host=tcp --port=//host:port for the tcp:// form the servers accept, so
backups failed. Read mysql_host the way the servers connect with it: tcp://host:port (a trailing
/database is dropped), unix:// as --socket and pipe:// as a named pipe.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The NexusDashboard-parity dashboard (dDashboardServer) and everything built on it on the experimental branch:
accounts, characters, properties and moderation tools, permissions shared with in-game slash commands, economy
reports, World 3D and property 3D views with client scenery, scheduled events (features, vanity changes, live
events, announcements, restarts), vanity files and events, the CDClient browser, the message inspector with saved
captures, chat filter tools, community challenges, live ops, the AI moderator helper, and the server-side changes
they need.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
