Commit Graph

23 Commits

Author SHA1 Message Date
Aaron Kimbrell
e0ccd3a016 Revert "feat(ugc): glitter flecks still by default"
This reverts commit 752c918311.
2026-09-28 22:31:22 -05:00
Aaron Kimbrell
fa563a558a feat(ugc): glitter flecks still by default
Glitter plastic has its flecks set in it; they catch the light as the view
moves, which LEGO-AnimUV can't do (it can only slide the texture). Drifting
flecks read as something flowing over the brick, so glitter_speed defaults
to 0: still flecks and no controllers. 1 and up still drift.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:22 -05:00
Aaron Kimbrell
7bba8344fc feat(ugc): glitter flecks (LEGO-AnimUV) and milky satin in made models
Glitter colors (Materials.xml type glitter, glitter_colors 114,117) go into
S21_Glitter_Model and, transparent, S21_GlitterAlpha_Model (shader_glitter,
default 21, LEGO-AnimUV). Their shapes get box-projected UVs, an
NiTexturingProperty with a stored 128 px mipmapped fleck texture
(NiSourceTexture + NiPersistentSrcTextureRendererData, as the client's own
env_ag_ocean-maelstrom.nif) and two NiTextureTransformControllers looping
the base map's translation (glitter_size, glitter_density, glitter_speed).
The shader lays the texture over the vertex color by its alpha and outputs
the vertex alpha, so transparent glitter blends as S01_Alpha does.

Satin colors (satin_colors, LEGO's opal colors) stay in S01_Alpha but get
satin_opacity and are whitened by satin_whiten.

NifFile reads the base map's scroll speed (uvScroll) from the controllers;
the icon draws still flecks, the UGC 3D view and the LXFML viewers moving
ones. stats.json counts the glitter groups. With shader_glitter 0 and no
satin colors the files are the same bytes as before (tested). Also keeps
glow_emissive for the icon (it was reset by the icon settings).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:21 -05:00
Aaron Kimbrell
462a152a66 feat(ugc): metal, brushed steel and glow shaders on by default
Tested in game: shader_metal 88, shader_brushed 89 and shader_glow 46 are
now the defaults, with brushed_colors defaulting to the drum lacquered
298,300,1002,1004 and transparent_colors to 129. Empty list settings take
the default; none turns a list off. The three shader ids set to 0 still
write live's all-plastic models byte for byte.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:20 -05:00
Aaron Kimbrell
0c5ef380d3 feat(ugc): color_brightness and transparent_colors
color_brightness (percent, default 100: unchanged) scales the models' vertex
colors, not the icons', for servers where the made models look brighter than
they should. transparent_colors names color ids drawn transparent whatever
Materials.xml says: 129 (Tr. Bright Bluish Violet with Glitter) has alpha
255 there, so it came out opaque; a named color gets transparent_opacity.
Both default to off, so models come out byte for byte as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:19 -05:00
Aaron Kimbrell
f3922b3bcb feat(ugc): brushed_colors, LEGO color ids drawn as brushed steel
The client's Materials.xml has no brushedSteel or matteSteel colors, so the
Brushed Steel shader (mapShaders 89) was never used. brushed_colors names
color ids to draw with it whatever their type (e.g. the drum lacquered
298, 300, 1002, 1004); a named color wins over the metal and glow colors.
Empty by default. The docs note that the client registers the brushed
reflection and noise textures itself (0x00453730), so the .nif needs none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:18 -05:00
Aaron Kimbrell
c39b7ee935 feat(ugc): opt-in metal and glow shader groups in made models
Player models are multishader (RenderComponent shader 100): the client
wraps each NiLODNode and draws it with the mapShaders id in its name.
With shader_metal, shader_brushed or shader_glow set, the opaque bricks
are split by look into S<id>_Metal_Model, S<id>_Brushed_Model and
S<id>_Glow_Model beside S01_Opaque_Model and S01_Alpha_Model, each with
every LOD level. Metal is LU Toolbox's metallic colors plus Materials.xml
types (shinySteel; brushedSteel and matteSteel for brushed), glow its
glow colors. Glow shapes get an emissive material (glow_emissive) and
their plain color, not the baked one. Transparent glow stays in S01_Alpha.

All off by default, which writes the same bytes as before (tested). Not
how live looked; models already made change only when made again.

The icon renderer reads the groups back by tag and draws glow at its
plain color and metal with a tinted reflection and highlight.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:18 -05:00
Aaron Kimbrell
9d76fe9550 feat(ugc): placed models from the UGC server without 3D services
A placed model's client (1.10.64, UGCUSE3DSERVICES=7:0) always loads its
blueprint's NIF and HKX (its BlueprintComponent sets renderUserGen and
physicsUserGen itself) and its LXFML, asks the world for each file's
manifest first and waits for the answer with no timeout.

With ugc_manifest=1 and the new ugc_manifest_models=1 (default 0) the world:
- leaves the models whose mesh the UGC server made out of the LXFML it
  sends when a property loads,
- answers their NIF with the UGC server's checksum, their LXFML with the
  stored LXFML's (worked out once and kept) and their HKX as not known,
- sends a model that isn't made its LXFML (once for the three requests),
  so the client builds it itself and no model is left waiting.

When the UGC server writes a model's mesh with a new checksum it sends
UGC_MODELS_MADE (a new master message, appended) to the master, which
passes it to every world; a world with that model placed sends its
players the new NIF checksum and NotifyClientUGCModelReady (game message
909, the blueprint id), so clients switch to the served mesh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:15 -05:00
Aaron Kimbrell
79235702b3 feat(ugc): answer the client's UGC manifest requests without 3D services
With UGCUSE3DSERVICES=7:0 (the client's default) the client asks its world for
a blueprint file's MD5 and size (REQUEST_UGC_MANIFEST_INFO, world 27) and then
downloads BrickModels/UserMade/<id % 1000>/<id>.<ext>.sd0. Layouts checked in
the 1.10.64 client: the request is a u64 blueprint id and a u8 resource type;
the answer (UGC_MANIFEST_RESPONSE, client 60) repeats them and adds a u8 valid,
the u32 size and the 16 byte MD5 of the inflated file, 37 bytes after the 0x53
exactly or the client drops it.

- dNet: WorldPackets::RequestUgcManifestInfo, ClientPackets::UgcManifestResponse
  (eUgcResourceType), with byte tests against the client's layouts.
- Database: ugc_file_checksums (per model or module combination and file) and
  ugc_modular_build.combination_id (migrations 89 / 72), GetUgcFileChecksum
  looks a blueprint up as a model, else as a build through its combination.
- UGC server: every download is also written as .sd0 (Sd0::Compress); workers
  hand the checksums back and the main thread stores them; old items get their
  sd0 icon and checksum, and builds their combination id, once at start-up, a
  few per tick; serves <dir>/BrickModels/UserMade/<bucket>/<id>.<ext>.sd0 under
  client_path, /<folder>/UserBrickModels and the root (.hkx 404).
- World: UgcManifest answers on the main thread with one indexed query per
  request; files not made yet are answered when they are (looked at again
  every 5 seconds), and a model in its quiet period is made right away. No
  worker threads, HTTP or file reads in the world.

Off by default (ugc_manifest=0): checked in game, the client then downloads
from http://127.0.0.1:80/lwoclient/UserBrickModels/ whatever its boot.cfg says
and logs the player out when it can't connect, so icons need the UGC server on
port 80 of each player's machine. docs/UgcServer.md has the details.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:14 -05:00
Aaron Kimbrell
36cefc1ede feat(ugc): brick data through the client's assets; colours from Materials.xml everywhere
The UGC server reads brickdb.zip and brickprimitives through DLU's
AssetManager (loose files first, then the client's packs, as the game
does), so packed clients and bricks added to a client work. It falls
back to loose files under res/ as before.

In the LU Toolbox palette (the default), a colour LU Toolbox doesn't
know but the client's Materials.xml has, such as one added to the brick
database, is drawn in its Materials.xml colour instead of black. Ids
neither knows stay LU Toolbox's black.

The dashboard's 3D viewers get the brick colours from the client's
Materials.xml (/api/bricks/materials.js) instead of a hardcoded copy, so
added colours show there too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:13 -05:00
Aaron Kimbrell
a5deacfc54 feat(ugc): icon pose math shared with the editor, model rotation parameters, assembled mesh endpoint
UgcIconPose holds the icon camera, model rotation (yaw/pitch/roll, YXZ) and
the crop to the projected bounds; RenderIcon uses it. The parameter list gains
the model's turn (defaults 0, so icons stay the same). POST /admin/assembly
returns a module combination's assembled .nif (turned by the build type's
AdditionalModelRotation), made on a worker and kept in a small cache.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:12 -05:00
Aaron Kimbrell
b4fa732047 fix(ugc): answer the client's built-in UGC path, and document how it downloads
In the 1.10.64 client tested, 3D services ignore boot.cfg's UGC and patch
server lines and ask http://127.0.0.1:80/lwoclient/UserBrickModels/...; the
UGC server now answers that path too (useful when it runs on port 80), and
the docs say to keep UGCUSE3DSERVICES=7:0 (the client builds the models from
the world's LXFML; verified that property models load) until it can.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:10 -05:00
Aaron Kimbrell
30da6575e3 fix(ugc): store meshes compressed, serve LXFML from the database, log client downloads
Each model took about 18 MB (the .nif uncompressed beside its .gz, the
unbaked .nif for the dashboard), so the 2 GB cap held about 110 models and
the server kept evicting and remaking them while clients got 408s. Only the
.gz files are stored now (the dashboard's copies are inflated when asked
for), and the previous version keeps only its icon, stats and .nif.gz.

The LXFML needs no making: its downloads come from the ugc row (kept in
memory for the last few), so they're never waited for or evicted. Every
client download is logged with its answer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:09 -05:00
Aaron Kimbrell
1a2758feab feat(ugc): icon light matched to the game, debounce, shared car icons, purge, icon editor
Brighter icons: a world light, a fill from the camera, a highlight, exposure
and contrast; with the defaults the icons' mean luminance matches the game's
own model icons (118 against 120 on a scratch set). Every icon parameter is
listed once (UgcIconParams: key, setting, range, default); the settings,
the dashboard's settings entries and the icon editor come from that list.
Presets per kind (player models, each car or rocket build type from
ModularBuildComponent) and overrides per model or module combination are in
ugc_icon_settings.

Saved models wait ugc_debounce_seconds (sharedconfig) after the owner's last
save before they're made (ugc.process_after); a client asking for one, the
owner leaving the world or a reset ends the wait.

Cars and rockets: one icon per combination of modules (sorted LOTs), shared
by every build of it; builds of a combination made already are marked made
right away, the client's per-blueprint downloads serve the shared files.

The dashboard can delete one item's files, purge by filter or all, preview
icons with any values on the UGC server (/admin routes, master password),
save presets and overrides, and draw a kind's icons again (icons only).

Migrations dlu/mysql/86 and dlu/sqlite/69. Not done yet: the dashboard
editor's lighting controls in the page script (routes are there), docs for
it, the empty-model state, /ugc?item= links, the shared fetch helper; the
storage size and property loading bugs are next.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:09 -05:00
Aaron Kimbrell
76cbbcc4ea feat(ugc): draw player models' icons from their generated NIF
The icon was rendered from a second mesh built only for it (the icon
renderer's color corrections, no variation, occlusion worked out again). It
is now drawn from the .nif just made, read back with NifFile at LOD 0, so it
shows exactly what the game shows: the color variation, the removed faces and
the lighting baked into the vertex colors (so it adds no occlusion of its
own). icon_correct_colors and icon_color_variation are gone; the camera and
light settings stay. Cars and rockets are drawn as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:09 -05:00
Aaron Kimbrell
175db95c98 feat(ugc): one shape per transparent brick, as LU Toolbox leaves them
LU Toolbox's Combine Transparent is off by default, so each transparent brick
is its own object and shape (the client can sort them). combine_transparent=1
joins them as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:06 -05:00
Aaron Kimbrell
7c5d69c1ed feat(ugc): make models like LU Toolbox does, within CPU and memory budgets
Parity with LU Toolbox's Process Model, Bake Lighting and icon renderer, with
its defaults as the settings' defaults:

- Colors from its LU palette (UgcPalette: LU colors, LDD colors mapped to the
  nearest LU one, unknown ones black), transparent bricks at 58.82% opacity, a
  brick transparent only when all of its materials are.
- Color variation: each brick's material gets its HSV value shifted in a 2.224
  gamma by up to 5% (times the color's own amount), from a random number of
  the model, brick and material, so reprocessing gives the same colors in
  every LOD. Icons get none, and the icon renderer's color corrections.
- LODs 0 and 2 with its distance logic, written as NiLODNode/NiRangeLODData
  like its exports and the game's own brick models, shapes divided at 65535
  vertices along the longest side like divide_mesh.
- Ambient occlusion like its AO-only bake: 64 rays per vertex, distance 5,
  after hidden surface removal, transparent bricks neither baked nor
  occluding, glow colors added.
- Icons from its icon scene: 50 mm lens at 53.4/19.5 degrees, sun of 2.5 at
  21/50.3 degrees with soft shadows, grey world light with occlusion; LOD 0's
  hidden surface removal and occlusion are reused for them.
- Optional ground plane for hidden surface removal; stats.json per model and
  the previous version's previews kept for comparing.

Budgets, applied live on config reload: max_cpu_percent (workers account
their thread CPU time and sleep to stay under it, long renders included),
worker_nice, max_memory_mb (jobs are estimated from their brick count and
wait until they fit), max_model_bricks and pause_hours. /status and the
traffic report show CPU, memory and throttling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:06 -05:00
Aaron Kimbrell
0de9decc7c fix(ugc): name crash dumps like the other servers
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:06 -05:00
Aaron Kimbrell
cb61fcf638 feat(ugc): write crash reports to dump_folder like world servers
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:06 -05:00
Aaron Kimbrell
69803448bb feat(master): report the UGC server in the server list and wait for it on shutdown
The server list now carries whether master starts the UGC server, whether it
is connected and the pid it was started as. Settings reloads reach it like
auth and chat, and shutdown waits for it. The UGC server sends its totals and
storage with its traffic reports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:05 -05:00
Aaron Kimbrell
d835e18c92 fix: follow main's log folders in the dashboard and UGC server
Servers now log to logs/<Server>/ (worlds to logs/WorldServer/<zone>/<clone>/) with the start time in the file name.
The dashboard and UGC server log the same way, and the System Log page and log search read the whole folder tree
instead of only the top of logs/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:05 -05:00
Aaron Kimbrell
6f334c2e3d feat(web): record HTTP requests by route with latency for the traffic report
Requests are counted by route pattern and status class with a latency
histogram (deferred requests until their answer goes out) and bytes sent.
The web server reports pending deferred requests and WebSocket clients, the
UGC server its worker threads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:01 -05:00
Aaron Kimbrell
abf1cc1d80 feat: UGC server that makes and serves player models' meshes and icons
A new server (dUgcServer, started by master with enable_ugc_server=1) that
takes unprocessed ugc and ugc_modular_build rows from the database and makes
what the client downloads with UGCUSE3DSERVICES: an optimized NIF (hidden
faces removed, ambient occlusion baked into vertex colors) and a 128px DDS
icon for player models, rendered by a software rasterizer from the client's
LDD brick primitives, and icons for cars and rockets assembled from their
modules per ModularBuildComponent/ModuleComponent. It serves them, with the
models' LXFML, over HTTP in the client's UGCC<dc>/3DOPTIMIZED and
IMAGE128DDS layout with .gz and .checksum files, and keeps its folder under
a size cap.

Processing state lives in ugc.is_optimized plus new processed_at,
process_attempts and process_error columns (and the same on
ugc_modular_build). ServiceType::UGC is appended. NifFile moves to dCommon
and records named node transforms for the modules' attach points.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:57 -05:00