Commit Graph

183 Commits

Author SHA1 Message Date
Aaron Kimbrell
c01828a3bb Merge exp/decoder-live-check: replica decoders checked on live captures, moving platform/pet/model wire fixes 2026-09-30 08:40:02 -05:00
Aaron Kimbrell
b42f1e09c8 docs(capture): replica decoding against live captures, in-game checks
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 08:39:33 -05:00
Aaron Kimbrell
1657ca006d feat(chat-filter): the server filters phrases in normal chat both ways
The client's word check before sending is a light one; the server is the full filter. In normal (whitelist) chat, block list phrases are now stopped even when each word is allowed, and allowed phrases (from the dashboard or a line with spaces in the allowed words file) let their words through together. The dashboard can allow phrases and its message test explains both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 08:27:14 -05:00
Aaron Kimbrell
9658014b4f docs(chat-filter): block list file, portable format, phrases
How to build blocklist.dcf from blocklist.txt and where it goes, the
version 3 layout and hash, what happens to old files, and blocked
phrases; README "This branch" line. issue 215

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 08:05:26 -05:00
Aaron Kimbrell
4d7bea6f06 test(capture): decoder coverage, replica read-back and generator check; docs
Every LU MessageType has a struct or is listed as never used by this server; every game message struct reads;
samples of each family and of sent game messages read back; replica constructions, updates and destructions written
by the server's own serializers read back. Generator finds constructors defined in the .cpp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 06:32:46 -05:00
Aaron Kimbrell
5d3adfc683 docs: activity matchmaking across worlds
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 04:29:59 -05:00
Aaron Kimbrell
9a4282e6b3 docs: properties stay out of live updates and world reloads
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 04:29:54 -05:00
Aaron Kimbrell
c6831ac80e fix(dashboard): light the UGC model view like the game
Metal was drawn with metalness 1 in the room environment at half strength with one
sun, so metal groups came out nearly black. The game's metal keeps the vertex
color (lit, plus a reflection tinted by it), so metal is now part metal with a
stronger reflection, the environment is brighter and a sky over ground fill
lights the sides away from the sun.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 03:50:37 -05:00
Aaron Kimbrell
19b6443fb4 fix(ugc): metal look only from the client's material types
LU Toolbox's metallic table also has colors the client's Materials.xml types
shinyPlastic, such as 131, the grey of many baseplates, so whole baseplates went
into S88_Metal_Model. Metal now comes from the Materials.xml types (shinySteel)
and the settings' colors only; the table still gives colors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 03:50:37 -05:00
Aaron Kimbrell
5265da7b2e docs: following a player across worlds
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 03:21:54 -05:00
Aaron Kimbrell
5658fb036f docs: world hot reload
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:51:21 -05:00
Aaron Kimbrell
230e5b4841 docs(dashboard): Network ports and servers on several machines
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 02:39:24 -05:00
Aaron Kimbrell
04f73a4e5b docs(cdclient): fdb copies and hot reload
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:26:37 -05:00
Aaron Kimbrell
f86f173cae fix(dashboard): Network page in six columns with chat in its own, boxes that leave room, links spread along each side
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:14:40 -05:00
Aaron Kimbrell
3a8838d463 feat(capture): replay bundles against a sandbox stack with a headless client
CaptureTool (built next to the servers) replays packet bundles and compares the answers:

- replay: per bundle a fresh sandbox folder with copied server binaries, rewritten settings
  (replay_sandbox=1, a new SQLite file inside the folder, ports from --port on, no dashboard),
  read back before anything starts; sandbox-setup runs inside it to apply migrations and make
  the replay account and the bundle's characters (setup mode); master is started, the stack is
  stopped as one process group and the folder deleted unless kept
- with replay_sandbox=1 every server refuses a database that isn't SQLite, isn't inside its
  own folder, or is replay_live_sqlite_path (Database::Connect); replay-target against a
  running server needs --i-know-this-is-not-a-sandbox
- the fake client splits the recording into connections, logs in and picks the character
  itself when the recording doesn't, fills in the target's account, session key and IDs,
  learns server-made object IDs from replica constructions by LOT, follows the recorded
  timing and waits for the answers a client waits for; the diff pairs answers by name (and
  constructions by LOT) and ignores fields that differ between runs
- import-live converts the 2014 live captures (folders of *_traffic.zip; pcaps and encrypted
  captures are left alone) into bundles, with secrets removed and CREATE_CHARACTER as setup
- anonymise makes local fixtures; docs/CaptureReplay.md describes capture, the bundle format,
  portability rules, the sandbox and the replay

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:34:34 -05:00
Aaron Kimbrell
355d997333 docs(dashboard): Network groups open in place; web clients by signed-in user
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
7c37fca9fb docs(dashboard): Performance page, slow frames, profiling and Tracy
How frame timing is measured and reported, the page, the API and connecting Tracy's viewer. Task 96.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
6758b32605 docs(dashboard): game text and languages
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:47 -05:00
Aaron Kimbrell
0787e651a6 docs: client system info and what each field is worth
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:09:22 -05:00
Aaron Kimbrell
1e5660512e refactor(chat): remove the chat server's web API
The dashboard's API covers what it offered (online players, teams, announcements) with API keys, so the
chat server no longer runs a web server; its settings show as unused on the settings page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:09:02 -05:00
Aaron Kimbrell
8948e2e81a fix(dashboard): Network links follow dragged boxes; Diagram or List, no Auto
Links leave and enter on the sides that face each other (top and bottom when one box is above the
other), so moving a box no longer leaves links ending in the air.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:07:27 -05:00
Aaron Kimbrell
6a59da1748 fix(dashboard): web clients sit right of the dashboard on the Network page
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:38:17 -05:00
Aaron Kimbrell
c9774173bb docs(dashboard): the Network page and traffic by peer
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:28:39 -05:00
Aaron Kimbrell
8a53562629 docs: chat histories and chat flags
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:23:23 -05:00
Aaron Kimbrell
1bdb4258a0 docs(ugc): LU Toolbox models stand up as the client reads them
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:13:45 -05:00
Aaron Kimbrell
b0fa440407 docs(ugc): LU Toolbox in Blender (processor=toolbox-blender)
Setup, settings, what niftools v0.1.1 needs changed to write LU
Toolbox's models, Blender 5.2, and timings and triangles against native
on five models.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 16:47:40 -05:00
Aaron Kimbrell
063361711b feat(dashboard): pick and compare the processor
Make again gets a Made by picker, /api/ugc/options lists the processor
choices and default, the settings page has the toolbox_* settings, and
/reprocessproperty takes native or toolbox-blender.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 16:47:40 -05:00
Aaron Kimbrell
5e12a6d4ec docs(packets): VERSION_CONFIRM build identifier layout
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 16:29:46 -05:00
Aaron Kimbrell
a742f20fdc docs(ugc): the Intel GPU ray backend (embree-gpu) and what it needs
docs/UgcServer.md: embree-gpu in the processing options, the build option
DLU_EMBREE_SYCL (a SYCL compiler: oneAPI's icpx or the open source DPC++), the
library it builds and loads, the run time it needs, the GPUs it supports and
embree_gpu_device; which backend covers which hardware.

Check: the section against the settings page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:29:08 -05:00
Aaron Kimbrell
db63fd2919 feat(ugc): ray_backend=embree-gpu traces on Intel GPUs with Embree's SYCL (optional build)
The third ray backend, so every machine has a library for it: Embree on x86
CPUs (embree), HIPRT on AMD and NVIDIA GPUs (hiprt), Embree through SYCL on
Intel Arc and Xe GPUs (embree-gpu).

- CMake option DLU_EMBREE_SYCL (off). dUgcServer/EmbreeSycl is a project of
  its own built by a SYCL compiler (DLU_SYCL_CXX; icpx through ONEAPI_ROOT or
  the path, or the open source DPC++'s clang++ through DPCPP_ROOT) as an
  external project: Embree 4.4 with EMBREE_SYCL_SUPPORT, linked statically and
  bound inside (-Bsymbolic, only its C functions exported, so it never meets
  the servers' own Embree), and the GPU kernels (nearest hit skipping a ray's
  triangle, any hit), into libdlu_embree_sycl next to the servers
- UgcRaysEmbreeGpu loads it the first time embree-gpu is asked for; one GPU for
  the process (embree_gpu_device picks it), the workers take turns, the
  occlusion rays in batches as for hiprt
- without the build, the library or a supported Intel GPU it falls back to
  embree and says why (the UGC server's log at start, --make-model on stderr)
- the option names, the settings page, the dashboard's picker, /reprocessproperty

Verified here: the default build and ctest; the SYCL build with the open source
DPC++ 7.1.0 (compiles, links against oneAPI's libsycl.so.9, exports only its C
functions); on this machine (no Intel GPU) it loads, finds no GPU and falls
back to embree. Not verified: tracing on an Intel GPU (none here).

Check: on a machine with an Intel Arc or Xe GPU and oneAPI, configure with
-DDLU_EMBREE_SYCL=ON and run UgcServer --make-model x.lxfml out embree-gpu;
the UGC tests then compare it with Embree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:29:08 -05:00
Aaron Kimbrell
a7defdb326 docs(ugc): hidden faces by the renders only; Embree and HIPRT the ray backends
docs/UgcServer.md: "Hidden faces" describes the renders from 42 directions
(hsr_resolution) and what they remove that LU Toolbox keeps, and why the path
traced version was removed; the parity table says so. "Processing options" has
ray_backend (embree, hiprt) and denoise, how options stored with builtin,
toolbox or fast are read, and what the tests compare the backends with.

Check: the sections match the settings page and the UGC page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:29:08 -05:00
Aaron Kimbrell
edba81ff7c refactor(ugc): Embree is required; the hand-written ray hierarchies are gone
Embree 4 (always built) replaces the UGC server's own bounding volume
hierarchies (the nearest-hit one and the occlusion rays' any-hit one), with no
fallback to them. ray_backend is embree (default) or hiprt (optional build;
Embree when it can't be used).

Settings, stored options and stats that say builtin still read: it is embree
(UgcRays::Parse, UgcProcessOptions::Parse). The dashboard's picker,
/reprocessproperty and --make-model offer embree and hiprt.

Tests: the backends are compared with Embree (hiprt when built), Embree against
rays whose hits are known, and the clutter's occlusion against what the old
hierarchy worked out (296 vertices summing to 114.5, 26 open, 183 dark); the
pinned model hashes are unchanged with Embree.

Check: ray_backend=builtin in an ini still starts and uses embree; the
settings page offers Embree and HIPRT.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:29:08 -05:00
Aaron Kimbrell
07154c58d2 refactor(ugc): remove the path traced hidden-face method; the renders are the only one
The path tracer that imitated LU Toolbox's Remove Hidden Faces took 17 to 100
times as long as the renders from 42 directions around the model, too heavy to
use. It goes with its settings (hsr_method, hsr_samples, hsr_bounces,
hsr_sample_spacing, hsr_min_points), its side by side tracing for GPUs and its
tests. The renders (UgcRender::VisibleFromAround) remove hidden faces as before
that method existed; their size is hsr_resolution (1024), and the memory
estimate counts their buffers again.

The hidden-face method is no longer a processing option: the dashboard's picker,
/reprocessproperty and --make-model take only the ray backend (the occlusion's)
and denoising. Options stored before (made_options, process_options,
ugc_process_runs, --make-model arguments) that name toolbox or fast still
parse; the word is skipped.

Check: the settings page has no hsr_method or path settings and has
hsr_resolution; /reprocessproperty embree toolbox still works (toolbox
ignored); models made with the defaults keep their hashes (UGC tests).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:29:08 -05:00
Aaron Kimbrell
b897681f67 docs(ugc): processing options (ray backend, hidden-face method, denoising)
docs/UgcServer.md gets "Processing options": the three settings and their
choices, how they compare with builtin/toolbox/off, the optional builds
(DLU_OIDN, DLU_HIPRT) and what they need, what each make records and where the
dashboard shows it; the migration, /reprocessproperty's options, the command
line's and the UGC page's buttons.

Check: the section reads right against the settings page and the UGC page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:29:08 -05:00
Aaron Kimbrell
ef3c74f715 feat(ugc): processing options per make, recorded per model with their times
Staff can make models again with other processing options than the UGC
settings' (ray backend, hidden-face method, denoising) and every make records
what made it, for comparing the options.

- migration mysql 99 / sqlite 82 (ugc_process_options): ugc.process_options
  (picked for the next make, cleared once made), ugc.made_options (what made
  the current files) and ugc_process_runs (every successful make: options,
  wall and CPU time, hidden faces', occlusion's and icon's time, bricks,
  triangles before and after)
- IUgc: ResetUgcModelProcessing and ResetPropertyUgcModelProcessing take the
  options; PendingModel carries them; RecordUgcModelRun, GetUgcRunSummaries;
  list entries have madeOptions and processOptions
- the UGC server applies a job's options over its settings and records the run
- /api/ugc/reprocess takes options ("embree fast oidn"); /api/ugc/options
  lists the choices, the settings' defaults and averages per combination
- /reprocessproperty [builtin|embree|hiprt] [toolbox|fast] [off|oidn], any
  order, all optional

Check: run the migration on MySQL and SQLite; /reprocessproperty embree fast
on a property, then the models' made_options and ugc_process_runs rows;
/api/ugc/options.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:29:08 -05:00
Aaron Kimbrell
7dc85692d9 docs: issue tracker: commit ids after rebasing issues 307, 1045, 1568, 1970 and 2016
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 10:53:00 -05:00
Aaron Kimbrell
873f5d89ba docs: issue tracker: issue 1568
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 10:52:54 -05:00
Aaron Kimbrell
1aa70c523b docs: issue tracker: issue 1045
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 10:52:54 -05:00
Aaron Kimbrell
f1f8785b26 docs: issue tracker: issues 307, 1970 and 2016
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 10:52:54 -05:00
Aaron Kimbrell
5853c96872 docs: issue tracker: issues 917 and 1189
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 09:39:17 -05:00
Aaron Kimbrell
e0169a0a63 docs: issue tracker: issue 1113
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 09:22:59 -05:00
Aaron Kimbrell
495d62e559 docs: issue tracker: issue 928
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 09:22:01 -05:00
Aaron Kimbrell
9e5a4b2ba0 docs: issue tracker: issue 1256
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 09:19:57 -05:00
Aaron Kimbrell
dceec20ac2 docs: issue tracker: issue 957
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 09:18:53 -05:00
Aaron Kimbrell
f7d21b22c2 docs: issue tracker: issue 611
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 09:12:32 -05:00
Aaron Kimbrell
154ff1f16f fix(minimap): venture vision shows again after logging in (issue 611)
Venture vision from an equipped item was sent while the character loaded, before the client's UI existed, so after a
login the minimap stayed empty until the item was equipped again (a world transfer keeps the UI, so it worked
there). The player's active venture vision effects are sent again once the player has loaded. Check: wear the
Venture Vision helmet, log out and back in: the minimap shows the icons.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 09:12:26 -05:00
Aaron Kimbrell
24d96f13df docs: issue tracker commit hashes as they are on the branch now
The table named commits by their hashes from before earlier rebases; each is now the current hash of the commit
with the same subject.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:41:32 -05:00
Aaron Kimbrell
4789711f82 docs(guilds): the login response's events turn guilds on
The client adds every event string of the login response as an enabled feature (AddFeatureGate from the login
response handler), so event_1..event_8 = guilds turns guilds on in a stock client; no client file needs changing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 06:56:05 -05:00
Aaron Kimbrell
4bdaeb8d3c docs: how DLU does guilds; guild commands and the Guilds page
docs/Guilds.md gains what DLU does with the client's guild system: the chat server as the authority, the world <-> chat
packets, the rank rules, the invite and name rules, the slash commands, the dashboard page and what is not done (a
member's name change reaches guildmates only with the next GUILD_DATA; reputation is 0). docs/Commands.md lists the guild
commands, docs/Dashboard.md the Guilds page, the guild names in the Review Queue and guild chat under chat_private.

Check: nothing in game (documentation only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 06:48:40 -05:00
Aaron Kimbrell
c99e2a719f docs: guilds in the 1.10.64 client (RE)
docs/Guilds.md: what the client's guild system needs, from Ghidra and the client's Scaleform UI. The feature gate
(FeatureGating "guilds", missing from the shipped cdclient), the UI calls and messages, what the character component
serializes, every guild packet's layout and what the client does with it (including the ones it has no handler for),
GUILD_DATA's member records, DisplayGuildCreateBox, the guild chat channel (the chat box sends /g), rank names and the
limits the client enforces. No live capture has guild traffic.

Check in game: nothing (documentation only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 06:48:40 -05:00