Commit Graph

374 Commits

Author SHA1 Message Date
Aaron Kimbrell
d670484b68 feat(dCommon): read cdclient.fdb in place through its hash buckets
FdbMappedFile maps a file read-only (CreateFileMapping/MapViewOfFile on
Windows, mmap elsewhere) and falls back to reading it into memory when
mapping fails. FdbReader reads the table and column headers from it and
looks rows up by their first column through the fdb's own buckets,
decoding every integer as little-endian with bounds checks, so the rows
never get copied out of the file.

Tests write small fdb files (collisions, text, int64, nulls) and read
them both mapped and from memory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:45 -05:00
Aaron Kimbrell
94b411a32e feat(combat): TacArc picks its targets in the client's order
The client's TacArcBehavior::Cast (0x00fb2d10) sorts the targets in the arc nearest first, or by weight when
distance_weight or angle_weight is set (sortWithWeights, 0x00f58cd0: distance_weight * (max range - distance) /
max range + angle_weight * (180 - angle) / 180, heaviest first). With use_attack_priority, SortByAttackPriority
(0x00f72900) then buckets them by GetAttackPriority, lowest first, keeping that order inside each bucket; only the
DestroyableComponent answers it, and an object without one counts as 1. DoHit keeps the first max targets.

Nothing else ranks targets: enemies are taken over nearer smashables only because their attack_priority (1) is
lower than most smashables' (10). use_attack_priority is off when a behavior does not set it (TacArcBehavior::
Initialize, 0x00f9b980), as before.

The server sorted by distance only and ignored the flag, so a one-target swing hit the nearest crate instead of the
enemy behind it. OrderTargets now does the client's ordering, with equal targets in ascending id order (the order
the client's id set hands them over in); the chosen targets are still written in ascending id order (issue 1045).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 21:44:48 -05:00
Aaron Kimbrell
c44af311ad feat(dashboard): client system info, as reported by the client
Account pages get a Client system info (as reported) card (client_sysinfo):
every description the account's client sent at login, newest first, with the
raw values, the memory text split into numbers and each field's caveat. The
address is only shown with logs_audit. The Client System Info page (Logs &
Health) shows the spread across players from each account's newest report:
Windows version, video card, memory buckets, processor count and client build,
marked as client-reported and approximate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:09:22 -05:00
Aaron Kimbrell
7bc41f2246 feat(db): client system info as reported by the client
client_sysinfo (migrations mysql 102, sqlite 85) keeps the system description
each account's client sent at login, exactly as sent, plus the physical memory
read from it. While nothing but the memory in use changes, the account's newest
row gets the new time and one more login; otherwise a new row starts. Log
pruning deletes rows not seen for a while (eLog::CLIENT_SYSINFO) and deleting
an account deletes its rows.

Tests on SQLite alone (dDatabaseSqliteTests) and in the MySQL parity tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:09:14 -05:00
Aaron Kimbrell
b67e0ba9ba feat(common): read the memory text the client sends at login
The login request's memoryStats is a run of short parts with no separators
(process working set and private bytes, memory load, physical memory, commit
limit, the 32-bit client's own address space, peaks). ParseMemoryStats splits
it into numbers; a text cut short keeps the parts before the cut.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:09:13 -05:00
Aaron Kimbrell
9b00e437e8 fix(dashboard): Network connections are one entry per player or per connection, not per address
A player on the servers' machine (or behind the same address as other players) was merged with the
servers' own links and everyone else there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:07:27 -05:00
Aaron Kimbrell
a824ff9914 feat(dashboard): drag the Network page's boxes around; links coloured against the busiest link
The layout the viewer makes is kept in the browser, with a Reset layout button. A link's colour now
compares it with the busiest link right now instead of its own recent peak, which made steady links
look saturated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:41:35 -05:00
Aaron Kimbrell
6a59da1748 fix(dashboard): web clients sit right of the dashboard on the Network page
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:38:17 -05:00
Aaron Kimbrell
e6b7677c1f feat(dashboard): Network page with a live traffic flow diagram
Next to Diagnostics (health_view): game and web clients, auth, chat,
worlds per zone (expandable), other reporting servers, master, the
dashboard and the UGC server, with a lane each way per link sized by
bytes/s, animated by packets/s and coloured by load against its own
5 minute peak. Estimated links (servers without a split) are dashed.
Clicking a box shows its links, busiest message types and a sparkline,
with a link to Diagnostics (which now takes ?server=). A connections
table lists each remote address; single clients can be drawn too.
Narrow screens get a list. Drawing stops while the tab is hidden.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:28:39 -05:00
Aaron Kimbrell
7e87f2a482 feat(dashboard): network summary, server detail and connection routes
The traffic topic now carries each server's rates with its split by
peer (null for servers that report none), link statistics and gauges.
New routes: /api/diagnostics/network, /network/server (message types
and a 10 minute series) and /network/connections (remote ends grouped
by address). Addresses need the new network_ips permission; without it
each is a salted token. They stay in memory from the last report only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:28:39 -05:00
Aaron Kimbrell
3b778065cf feat(traffic): send the peer split and connections in SERVER_TRAFFIC
Two optional sections at the end of the report, each after a marker
byte: every second's packets by peer with its HTTP requests from and to
other servers, and the busiest remote ends with the rest summed.
Reports without them still read (older servers), and older readers stop
before them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:28:39 -05:00
Aaron Kimbrell
9e8f161dd8 feat(traffic): count traffic by peer and by connection
Every server now splits its packet counts by peer: its own connections
(players on auth and worlds), its master link, or other servers (the
worlds on chat, every server on master, a world's chat link, which is
now counted too). HTTP requests carrying X-Darkflame-Server count as
another server's, the dashboard counts its own requests to the UGC
server, and each HTTP client address is counted. The report also gets
each RakNet connection's statistics (worlds name the player on it),
trimmed to the 32 busiest with the rest summed. Counting stays on the
main loop, except the dashboard's UGC fetches, which only touch the
locked recorder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:28:39 -05:00
Aaron Kimbrell
f3a49991d4 feat(dashboard): guild, team and whisper chat histories
Guild chat per guild (/chat_log/guild/<id>, from the guild's card), team chat by
team (/chat_log/teams) and a character's whispers by conversation
(/characters/<id>/whispers, chat_dms; opening one is audited), paged on the
server. The Chat Log filters by account and time range, shows the conversation
around a message, links to its history and marks flagged messages; messages can
be picked (shift-click for a range) to flag. The character page links to all of
them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:23:23 -05:00
Aaron Kimbrell
af54ca282f feat(db): chat log guild, team and filter result; chat flags
chat_log gets guild_id, team_id and filtered (migrations mysql 100, sqlite 83),
and the chat log queries take a time range, a guild, a team, a conversation
between two characters and paging back by id. Whispers are their own visibility
switch (includeWhispers), apart from team and guild chat (includePrivate).
Whisper partners and teams that talked are listed with counts.

chat_flags, chat_flag_messages and chat_flag_events (mysql 101, sqlite 84) hold
flagged chat with a copy of the messages around it, its status, note, linked
player report and history. Open flags are in the dashboard snapshot.

Tests on SQLite alone (dDatabaseSqliteTests) and in the MySQL parity tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:23:22 -05:00
Aaron Kimbrell
c01824e74d test(ugc): pin where the UGC server's own .nif vertices read back
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:13:45 -05:00
Aaron Kimbrell
602acc0fe7 fix(nif): leave out a root node's rotation and translation
The client's render component sets the object's own position and
rotation on the root node it loads, over the stored ones, so a root's
turn never shows in game (its scale stays). NifFile now reads roots the
same way. LU Toolbox's .nif (root turned 90 degrees about X, the
NiLODNodes turned back) now stands up in icons and the dashboard's
views; every game .nif has an unturned root and reads as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:13:45 -05:00
Aaron Kimbrell
6208724e6a feat(ai): enemies' paths stop at movement blockers
MovementAIComponent::SetDestination cuts an enemy's path (chasing, tethering,
wandering) where it first walks into a wall its collision group can't cross,
so enemies no longer walk through the Sentinel camp walls, the Crux Prime and
property navmesh carvers, or the enemy blocking volumes. With nothing left to
walk the enemy stays put. Patrols along a level path, and movers without combat
AI, are left alone.

Adds a scenario test: an enemy chasing across an enemy only volume and across a
carver stops in front of it; the clear threat wall is a trigger, not a wall.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 17:52:02 -05:00
Aaron Kimbrell
425b913848 feat(physics): movement blockers the server's movers can't walk through
Enemies are moved along navmesh paths without collision, so a wall only stops
them if their path stops at it. dpWorld now keeps a list of such walls, each
with the collision filter it blocks with, and cuts a path where it first walks
into one (a little short of the wall; a mover inside one can walk out).

Which objects block comes from data (dpMovementBlockers::BlockingFilter):
- navmesh carvers (navmesh_carver in the level config, which the client reads
  with add_to_navmesh and carver_only) block every mover;
- solid objects whose collision group touches enemies and not players (group 18,
  e.g. "FV - Enemy Blocking Volume", "PR - Pet Blocker") block what that group
  touches.

dpShapeBox gains SegmentEntry, a segment test against the rotated box.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 17:52:02 -05:00
Aaron Kimbrell
6ae6eac64e feat(ugc): LU Toolbox worker in a headless Blender
UgcToolbox runs LU Toolbox itself in an external Blender that stays up
and makes one model after another (dlu_toolbox_worker.py runs
LU-Toolbox-Standalone's steps: import, Process Model, Bake Lighting,
niftools export). JSON lines over its stdin and original stdout; it is
restarted after a crash, a timeout, 100 models or new settings, runs at
worker_nice with toolbox_threads, and its CPU time is charged to the
worker, which pauses it (SIGSTOP) while the CPU budget is overdrawn.
ProcessModelToolbox reads LU Toolbox's .nif back, counts its triangles
and draws the icon from it. Tests: option parsing, the fallback, the
protocol framing, the worker with a stand-in Blender (crash, hang,
failure, give-up), and LU Toolbox itself when DLU_TEST_TOOLBOX_* are set.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 16:47:40 -05:00
Aaron Kimbrell
05d18f56e4 feat(net): VERSION_CONFIRM carries the server's build identifier
After serviceType the reply now sends major, minor, patch, a flags byte
(build kind in bits 0-1, dirty in bit 2), the first 32 bits of the commit
hash and a u16 length-prefixed build string, instead of the stale fixed
ASCII "0.1.3". unknown stays "DLU3". The 1.10.64 client reads only
netVersion and serviceType and never checks the length, so the extra
bytes are ignored. The build string is optional on read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 16:29:46 -05:00
Aaron Kimbrell
eeb91fa6df test(loot): a luz spawner path's smashable_loot_matrix reaches the entity
Live GF Large Crates (LOT 1859, path CrateMast) and FV small white shrines
(LOT 3141, path gate_statue_quickbuild) dropped 1-point powerups outside
their template matrices, from the path's smashable_loot_matrix=1:29 with
smashable_loot_matrix_set=7:1. DLU already carries a spawner path's
waypoint config into the spawned entity's settings; this pins it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 13:06:37 -05:00
Aaron Kimbrell
b8a7ac897d fix(quickbuild): take precondition items when the build starts, give them back on cancel
A quickbuild's HasItem preconditions took their items only when the build
completed and never gave them back. Live took them when the build started
and gave them back when it was cancelled: the FV Stone Warrior pedestal
(LOT 8551, precondition 99: 5 of LOT 6194) took the items at Building 5
times and added them back with the loot source Quickbuild on each of the 3
cancels in the live captures.

Preconditions now report their item costs instead of removing items while
checking. A quickbuild takes them at the start, gives them back on cancel
or a reset during the build, keeps them on completion, and gives them back
when the builder leaves the world mid-build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 13:06:37 -05:00
Aaron Kimbrell
616ca46e95 fix(loot): activity reward coins use the reward's ChallengeRating level
ActivityRewards coins were always read from CurrencyTable npcminlevel 1.
Live used the reward row's ChallengeRating as the npcminlevel, and level 1
when the currency index has no row for that level:
- FV foot races (ChallengeRating 4, index 1) gave 36 and 48 coins, which
  only level 4 (30-50) fits; level 1 is 3-5.
- Frakjaw's chest (activity 58, ChallengeRating 6) gave 250 each to a team
  of 2: its currency indices 123-126 only have a level 6 row (500), so DLU
  gave nothing.
- Quickbuilds, wishing wells and chests have ChallengeRating 1; survival
  and the shooting galleries have ratings with no row and keep level 1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 13:06:37 -05:00
Aaron Kimbrell
db63fd2919 feat(ugc): ray_backend=embree-gpu traces on Intel GPUs with Embree's SYCL (optional build)
The third ray backend, so every machine has a library for it: Embree on x86
CPUs (embree), HIPRT on AMD and NVIDIA GPUs (hiprt), Embree through SYCL on
Intel Arc and Xe GPUs (embree-gpu).

- CMake option DLU_EMBREE_SYCL (off). dUgcServer/EmbreeSycl is a project of
  its own built by a SYCL compiler (DLU_SYCL_CXX; icpx through ONEAPI_ROOT or
  the path, or the open source DPC++'s clang++ through DPCPP_ROOT) as an
  external project: Embree 4.4 with EMBREE_SYCL_SUPPORT, linked statically and
  bound inside (-Bsymbolic, only its C functions exported, so it never meets
  the servers' own Embree), and the GPU kernels (nearest hit skipping a ray's
  triangle, any hit), into libdlu_embree_sycl next to the servers
- UgcRaysEmbreeGpu loads it the first time embree-gpu is asked for; one GPU for
  the process (embree_gpu_device picks it), the workers take turns, the
  occlusion rays in batches as for hiprt
- without the build, the library or a supported Intel GPU it falls back to
  embree and says why (the UGC server's log at start, --make-model on stderr)
- the option names, the settings page, the dashboard's picker, /reprocessproperty

Verified here: the default build and ctest; the SYCL build with the open source
DPC++ 7.1.0 (compiles, links against oneAPI's libsycl.so.9, exports only its C
functions); on this machine (no Intel GPU) it loads, finds no GPU and falls
back to embree. Not verified: tracing on an Intel GPU (none here).

Check: on a machine with an Intel Arc or Xe GPU and oneAPI, configure with
-DDLU_EMBREE_SYCL=ON and run UgcServer --make-model x.lxfml out embree-gpu;
the UGC tests then compare it with Embree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:29:08 -05:00
Aaron Kimbrell
edba81ff7c refactor(ugc): Embree is required; the hand-written ray hierarchies are gone
Embree 4 (always built) replaces the UGC server's own bounding volume
hierarchies (the nearest-hit one and the occlusion rays' any-hit one), with no
fallback to them. ray_backend is embree (default) or hiprt (optional build;
Embree when it can't be used).

Settings, stored options and stats that say builtin still read: it is embree
(UgcRays::Parse, UgcProcessOptions::Parse). The dashboard's picker,
/reprocessproperty and --make-model offer embree and hiprt.

Tests: the backends are compared with Embree (hiprt when built), Embree against
rays whose hits are known, and the clutter's occlusion against what the old
hierarchy worked out (296 vertices summing to 114.5, 26 open, 183 dark); the
pinned model hashes are unchanged with Embree.

Check: ray_backend=builtin in an ini still starts and uses embree; the
settings page offers Embree and HIPRT.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:29:08 -05:00
Aaron Kimbrell
07154c58d2 refactor(ugc): remove the path traced hidden-face method; the renders are the only one
The path tracer that imitated LU Toolbox's Remove Hidden Faces took 17 to 100
times as long as the renders from 42 directions around the model, too heavy to
use. It goes with its settings (hsr_method, hsr_samples, hsr_bounces,
hsr_sample_spacing, hsr_min_points), its side by side tracing for GPUs and its
tests. The renders (UgcRender::VisibleFromAround) remove hidden faces as before
that method existed; their size is hsr_resolution (1024), and the memory
estimate counts their buffers again.

The hidden-face method is no longer a processing option: the dashboard's picker,
/reprocessproperty and --make-model take only the ray backend (the occlusion's)
and denoising. Options stored before (made_options, process_options,
ugc_process_runs, --make-model arguments) that name toolbox or fast still
parse; the word is skipped.

Check: the settings page has no hsr_method or path settings and has
hsr_resolution; /reprocessproperty embree toolbox still works (toolbox
ignored); models made with the defaults keep their hashes (UGC tests).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:29:08 -05:00
Aaron Kimbrell
07dbf30dc8 feat(ugc): ray_backend=hiprt traces on the GPU with HIPRT and Orochi (optional build)
HIPRT (MIT) behind the CMake option DLU_HIPRT (off): its headers come from its
SDK (HIPRT_ROOT, else ROCm's /opt/rocm) and are copied next to the servers; its
library is loaded when first used (hiprtew), as HIP or CUDA are by Orochi
(MIT, fetched pinned by hash; CUDA when its toolkit is found). The trace
kernels (nearest hit skipping the triangle a ray leaves, any hit) are compiled
the first time and kept in cache/hiprt. One GPU context for the process
(hiprt_device picks the GPU); the workers take turns on it. When HIPRT, the
GPU or a scene's upload fails, Embree is used instead, and the UGC server logs
why at start.

For a GPU the rays go in batches (UgcRays::Scene gets batch queries; the CPU
backends answer them a ray at a time):
- hidden faces: with a batch backend the paths are traced side by side, a
  bounce at a time (the path code split into Start, Scatter and Bounce, the one
  by one tracing unchanged); the same paths with the same random numbers, so
  the same triangles are decided (tested with builtin side by side)
- the occlusion bake and the denoised icons' traced occlusion always ask in
  batches (the same rays, the same results)

Its symbols are hidden: the servers export theirs (-rdynamic), and HIPRT's
library, which has an Orochi of its own, would otherwise call ours.

Check: configure with -DDLU_HIPRT=ON on a machine with ROCm (or HIPRT's SDK)
and an AMD RDNA or NVIDIA GPU; UgcServer --make-model x.lxfml out hiprt; the
UGC tests (hits, hidden faces and occlusion against builtin); a build without
it leaves everything as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:29:08 -05:00
Aaron Kimbrell
2e2e8153e2 feat(ugc): denoise=oidn traces icons' occlusion per pixel and denoises it (optional build)
Intel Open Image Denoise 2 (Apache-2.0) behind the CMake option DLU_OIDN (off):
an installed OIDN is used when found, else Intel's release package (pinned by
hash) is downloaded and its libraries copied next to the servers.

A denoiser only removes noise that differs from pixel to pixel; the icons'
occlusion comes from the bake, per vertex, which it leaves as it is (checked:
white noise 0.17 -> 0.006 relative spread, per-vertex blocks unchanged). So
with denoise=oidn a model's icon is drawn from model.noao.nif (its colors
before the bake) with its occlusion traced per pixel of the supersampled image
(denoise_samples rays, default 4, with the bake's distance and strength and the
ray backend), box filtered and denoised at the icon's size, guided by the colors
and normals. The model keeps its baked occlusion. Icons drawn again from stored
files use the stored model.noao.nif the same way.

OIDN works on a thread of its own; its time is charged to the job's thread
(UgcThrottle::Charge), so the CPU budget and the recorded CPU time include it.

Check: configure with -DDLU_OIDN=ON; UgcServer --make-model x.lxfml out oidn
and compare its icon.png with one made with off; an OFF build leaves icons as
they were.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:29:08 -05:00
Aaron Kimbrell
ef3c74f715 feat(ugc): processing options per make, recorded per model with their times
Staff can make models again with other processing options than the UGC
settings' (ray backend, hidden-face method, denoising) and every make records
what made it, for comparing the options.

- migration mysql 99 / sqlite 82 (ugc_process_options): ugc.process_options
  (picked for the next make, cleared once made), ugc.made_options (what made
  the current files) and ugc_process_runs (every successful make: options,
  wall and CPU time, hidden faces', occlusion's and icon's time, bricks,
  triangles before and after)
- IUgc: ResetUgcModelProcessing and ResetPropertyUgcModelProcessing take the
  options; PendingModel carries them; RecordUgcModelRun, GetUgcRunSummaries;
  list entries have madeOptions and processOptions
- the UGC server applies a job's options over its settings and records the run
- /api/ugc/reprocess takes options ("embree fast oidn"); /api/ugc/options
  lists the choices, the settings' defaults and averages per combination
- /reprocessproperty [builtin|embree|hiprt] [toolbox|fast] [off|oidn], any
  order, all optional

Check: run the migration on MySQL and SQLite; /reprocessproperty embree fast
on a property, then the models' made_options and ugc_process_runs rows;
/api/ugc/options.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:29:08 -05:00
Aaron Kimbrell
06be40471b feat(ugc): settings for the ray tracer, hidden-face method and denoising
New UGC settings (ugcconfig.ini and the dashboard's settings page):

- ray_backend: builtin (default), embree or hiprt (falls back to embree when
  the build or machine can't); the hidden faces' paths and the occlusion rays
- hsr_method: toolbox (default, LU Toolbox's paths) or fast (the renders from
  around the model), and hsr_fast_resolution (1024) for the fast one
- denoise: off (default) or oidn (icons; off until a build has it)

UgcProcessOptions (dCommon/UgcKeys.h) names the choices for everything that
passes them on ("embree fast oidn", any order, left out: the setting's).
UgcJobs::ApplyOptions puts a choice over the settings and MadeWith says what a
make used after fallbacks; a made model's stats.json records it (rays,
hsrMethod, denoise). UgcServer --make-model and --make-modular take the
choices after the folder and print the CPU time and what made it.

Check: the defaults make the same files as before; the settings page shows the
four settings under UGC; UgcServer --make-model model.lxfml out embree fast.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:29:08 -05:00
Aaron Kimbrell
00607a20a6 feat(ugc): the fast hidden-face method back, next to LU Toolbox's
UgcHsr::Options::method picks how hidden faces are found: toolbox (LU Toolbox's
paths, the default, unchanged) or fast, the test the UGC server used before
"hidden faces removed as LU Toolbox's Remove Hidden Faces decides them",
restored unchanged as UgcRender::VisibleFromAround: the opaque mesh rendered
from 42 directions around the model (Options::fastResolution pixels square,
1024 as before) and the triangles that show in none removed. It is much
faster, but it also removes faces seen only by bounced light (interiors,
recesses), which LU Toolbox keeps. Nothing sets it yet.

Check: the UGC tests (the fast method removes a box seen only through a
chimney, which the paths keep; its files are the same every time).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:29:08 -05:00
Aaron Kimbrell
57110a7081 feat(ugc): rays behind a backend interface, with Embree as a second backend
UgcRays::Scene answers the two ray queries the UGC server makes: the nearest
hit for the hidden faces' paths (never the triangle a path leaves) and any hit
for the ambient occlusion rays. Two backends:

- builtin: the two hierarchies the queries had before, moved unchanged (each
  built the first time it is asked), so the files made are the same bytes
- embree: Embree 4 on the job's thread (a device per worker thread, no threads
  of its own, so its time counts in the CPU budget), watertight, the skipped
  triangle filtered out

UgcHsr::Options::rays and UgcRender::AoOptions::rays pick the backend; both
default to builtin, and nothing sets them yet.

Check: the UGC tests (builtin's files keep their hashes; UgcRays tests compare
embree's hits, hidden faces and occlusion with builtin's).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:29:08 -05:00
Aaron Kimbrell
585e2a6ace fix(inventory): a removed item is freed by its inventory, not by itself (issue 1568)
Item::RemoveFromInventory ended with `delete this`, so every caller that still used the item afterwards (SetCount(0)
returning into its caller, loops that remove several items, proxies purged while their parent is handled) touched
freed memory. The inventory now takes the removed item and frees it at the inventory component's next update, or
with the inventory.

Check in game: sell, drop, delete, trade, mail and use up stacks (including the last of a stack); unequip and
remove an item set piece and a proxy-bearing item (rocket, modular car); donate items; nothing crashes and the
inventory shows the right counts after relogging.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 10:52:54 -05:00
Aaron Kimbrell
3300784cd9 fix(combat): TacArc targets are read and written in the client's order (issue 1045)
The client's TacArcBehavior::DoHit (0x00fb10c0) writes the closest max-targets ids from a set, so ascending and
each once, then the action data per id in that same order; DoUnserializeBS (0x00fb26a0) reads them the same way and
skips empty ids. The server handled the targets in list order and skipped ids whose object it could not find
without reading their action data, so every later target read the wrong bits (several pirates under a Doom Slicer).
Handle now reads the action for every listed id in ascending order, and the server's own casts write ids and actions
in ascending order after picking the closest targets.

TacArcBehavior::Cast (0x00fb2d10): a picked target that passes the filter gets the action with no TacArc data (the
server's own cast now calculates that action instead of handling it); otherwise the target is dropped, and an arc
measured from the target's position writes nothing.

Check in game: Doom Slicer and multi-target katanas on groups of pirates/admirals damage each of them; apes still
take damage during their stun; enemies with arc attacks (apes, Maelstrom horsemen) still hit players.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 10:52:54 -05:00
Aaron Kimbrell
5dba7d5136 fix(scripts): Spider Queen rain of fire hits two spots in each outer ring (issue 2016)
The live boss script's rain of fire takes every target of the first ROF target group and ROFImpactCnt (2) different
random targets of each other group. The server took one per group, so the rain was sparser than live. The impacts are
now picked like the live script (without repeats inside a group).

Check in game: AG Spider Queen stage 3; the rain of fire lands on the centre ring and on two spots in each outer ring.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 10:52:54 -05:00
Aaron Kimbrell
300932c657 fix(scripts): Spider Queen rapid fire sweeps the three zone target groups around a player (issue 2016)
The live boss script tracks which arena zone volume (Zone1Vol..Zone8Vol, TeleVol for the default Zone3Vol) each
player last entered, and its rapid fire picks a random player, takes the three RFS target groups around that zone,
sorts each by the targets' CWOrder (CWOrder2 when the sweep crosses between zones 8 and 1), clockwise or
counter-clockwise at random, drops the first and last target of the middle group (shared with its neighbours),
turns with skill 1480 at the fourth target and fires 1394 at every target in turn, playing attack-shoot-right or
attack-shoot-left. The server shot a single random target with the single-shot animation, and the AG property zone
never subscribed the boss to the zone volumes. The zone now registers the volumes (retrying until they are spawned)
and the boss builds the sweep like the live script.

Check in game: AG Spider Queen stage 2; the rapid fire is an arc of many shots sweeping across the arena near the
player, left or right, and follows the player to other parts of the arena; after teleporting it starts from the
default side.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 10:52:54 -05:00
Aaron Kimbrell
990cc74aa0 fix(scripts): Spider Queen specials follow her stage and wait out the smash (issue 1970)
Every hit started both the rapid fire shooter and the rain of fire timers, from stage 1 on, so the two specials
ran in every stage and on top of each other, each turning the boss's AI off and on again under the other's
animations. They also wrote the "stoppedFlag" that the no-players-around attack stop uses, which could leave her
stopped for good. As the live script: after she comes back down, a skill manager fires the rapid fire shooter in
stage 2 and the rain of fire in stage 3 only, again 10 to 15 s after each ends; for 3.1 s after her melee smash
(skill 322) a due special waits and fires when the smash ends. The rain of fire keeps her from attacking until
its last impact.

Check in game: Spider Queen fight: no specials before the first spiderling wave; stage 2 only rapid fire, stage 3
only rain of fire; she never freezes in the smash animation and keeps attacking after each special.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 10:52:54 -05:00
Aaron Kimbrell
4900f11cf1 fix(scripts): the Spider Queen screams from the mountain when a spiderling dies (issue 307)
The zone script never handed the boss her landing target and scream emitter: ZoneAgProperty::ProcessGroupObjects
was empty and nothing answered the boss client script's "QueryZoneScript" event. As the live scripts: the boss asks
the zone ("RetrieveZoneData"), the zone stores the first object of Land_Target and Spider_Scream on her as
LandingTarget and ScreamEmitter (looking again every 0.3 s until they are spawned), and each spiderling death sends
NotifyClientObject "EmitScream" with the emitter, which the boss's client script plays as the scream. The landing
skill and camera shake now come from the landing target, not the boss.

Check in game: AG Spider Queen (property or instance): kill a spiderling: the scream plays from the mountain; when
she comes back down, the landing blast hits around the landing spot and the camera shakes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 10:52:54 -05:00
Aaron Kimbrell
3a14dc4c9f chore: stop tracking Python bytecode
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:39:37 -05:00
Aaron Kimbrell
5cd8b183cc test(zone): use neutral names and IDs in the zone file tests
The zone file tests' sample zones now use generic world IDs, file names
and path names instead of ones taken from real files. What they test is
unchanged.

Check in game: nothing (test only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:39:37 -05:00
Aaron Kimbrell
f73f4da0e4 fix(level): split a scene object's config as the client does
LDF_FROM_STRING (1.10.64, 0x010fa220) splits an object's config at
every comma and line break, leaves out empty entries and entries with
no '=', and sets each key=value it finds. The reader split at line
breaks only and parsed every piece, so an object with no config got an
empty entry of unknown type, and a value with a comma would have stayed
whole.

Live scene files are unchanged; objects in older client files with an
empty config no longer get the empty entry.

Check in game: every world spawns its objects as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:39:37 -05:00
Aaron Kimbrell
272fafc886 test(zone): read every zone, scene and terrain file the client reads
ClientZoneFilesTests now
- reads sd0 files instead of skipping them: some clients' loose sd0
  files hold what followed the file in its pack, and the file is the
  first sd0 stream, up to the first chunk that does not fit;
- reads the terrain (.raw) of every zone of version 30 or newer, once
  per terrain file shipped by several clients;
- skips a damaged file (by the MD5 of its bytes) only when it does not
  read, so only the files the client cannot read either are left out:
  a scene file that claims 81 objects and ends after the first, and six
  version 30 terrain files with a width x width scene map per chunk
  where the client reads one byte (RAWReadSceneMap, 1.10.64) and so
  misreads every chunk after the first.

Every other zone, scene and terrain file of every client on disk reads,
in about 80 seconds.

Check in game: nothing (test only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:39:37 -05:00
Aaron Kimbrell
93c7b1948f fix(terrain): read terrain files before version 32 as the client does
The client reads a terrain chunk of a file older than version 32
(1.10.64):
- its color map as width x width BGRA pixels, keeping the
  (width - 1) x (width - 1) before the last row and column, as RGBA
  (RAWReadColorandLightMaps); the reader kept all the pixels as
  written;
- its texture blend map's pixels as BGRA, kept as RGBA (0x0103aaf0);
- before version 31 no scene map, only a byte: the client's scene map is
  all scene 0 (RAWReadSceneMap); the reader had none.

A chunk of width or height 0, which the client reads (no heights, no
color map), no longer fails the whole file.

Live terrain files are version 32, so they read as before.

Check in game: nothing to check (no live terrain changes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:39:37 -05:00
Aaron Kimbrell
695bcf2352 fix(level): change scene object config as the client does while loading it
ReadLvlObjectData (1.10.64, 0x0103ba20) changes an object's config as it
loads it:
- before version 47 a respawn time is in milliseconds: a float over 100
  or any u32 becomes that many seconds, as a float;
- a model spawner (LOT 176 with spawntemplate 14) gets its subkey as its
  blueprintid unless it has one, DisableModelBehaviors true unless set,
  and preventRenderWrapping true;
- before version 44 an object with sceneIDOverrideEnabled gets
  sceneLayerIDOverride 0.

LevelFile now makes the same changes. On live worlds 1906 spawners in
114 scene files of versions 45-46 have a u32 respawn time; the world
already divided those by 1000 but as an integer, so they now keep their
fraction (a 2500 ms respawn is 2.5 s, not 2 s). No live scene has a
model spawner; 62 objects of version 43 scenes get sceneLayerIDOverride.

Check in game: enemies and smashables respawn after the same time as
before on Forbidden Valley, Avant Gardens and Gnarled Forest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:39:37 -05:00
Aaron Kimbrell
458fb601c9 fix(level): find a scene file's chunks and sections as the client does
SceneLoader::ReadLvlFile (1.10.64):
- chunked files: the file info chunk starts the file and gives where
  the environment, object and particle chunks start, 0 for none
  (DoLvlChunk, ReadLvlChunk1000); the reader walked the chunks one after
  another by their sizes instead;
- older files: the editor settings are a u32 size and that many bytes
  (SceneLoader::ReadEditorSettings, skipped by size); the reader took
  them for a u32 and then a count of 12 byte points;
- older files before version 3 have no objects for the client ("Level
  file is unsupported").

LevelFile now does the same. Every scene file on disk reads the same
objects as before.

Check in game: every world spawns its objects as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:39:37 -05:00
Aaron Kimbrell
96d73c30a4 fix(level): read scene objects as the client does
ReadLvlObjectData (1.10.64, 0x0103ba20) reads a scene object's last
field, from version 7 on, as a render technique count, and when it is
not 0 a 64 byte block and 133 bytes per technique (a 64 byte name, a
u32, a u8 and 16 floats) follow. The reader took the count for an
unknown u32 and read the next object from inside the techniques, and
read the u32 for every version. It also left the node type and glom ID
uninitialized when a file has none (or, for the node type, one outside
0-10); the client uses 1 for both.

The count is now SceneObject::renderTechniqueCount, the techniques are
skipped, and the defaults are the client's. Every scene file on disk
has no render techniques, so what they read is unchanged.

Check in game: every world spawns its objects as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:36:35 -05:00
Aaron Kimbrell
13a3d18789 fix(zone): adjust path values as the client does while reading them
LevelPath::FromBuffer (1.10.64) changes two values as it reads a path:
- a property path's reputation multiplier outside 0-1000 becomes 0;
- a camera waypoint's FOV in a path older than version 11 is divided
  by 1.25.

The reader kept both as written. Only camera paths of older client
files (versions 8-10) change; no live zone has either case.

Check in game: nothing to check (no live zone changes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:36:35 -05:00
Aaron Kimbrell
83c2136e9a fix(zone): spawner and camera paths have their fields only from version 4
The client reads a spawner path's fields (LOT, respawn time, counts,
object ID) and a camera path's next path and rotate-player flag only
when the path is newer than version 3 (LevelPath::FromBuffer, 1.10.64:
pathVersion > 3). The reader read them for every version, which would
misread an older spawner or camera path from there on.

No zone file on disk has a spawner or camera path older than version 4,
so what they read is unchanged.

Check in game: nothing to check (no live zone changes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:36:35 -05:00
Aaron Kimbrell
17425c1e44 fix(zone): read a zone's paths as the chunk the client reads
The client reads a zone file's paths as a chunk of its own
(LuzFile::ReadLUZFile copies the u32-sized chunk, ReadLUZPaths reads
it, 1.10.64) and drops every path when it refuses any of it:
- a chunk version of 2 or more, or 10000 paths or more (ReadLUZPaths);
- a path of version 19 or more, 10000 waypoints or more, or a
  waypoint with more than 99 name/value pairs (LevelPath::FromBuffer).

The reader read the paths straight from the file, trusting the chunk to
be well formed. It now reads the chunk by its size, reads the paths
from it with the client's limits, and when one is refused the zone has
no paths (logged), while the rest of the zone file still reads.

Every zone file on disk reads the same paths as before.

The unit tests' sample zones now give the path chunk its real length.

Check in game: moving platforms, NPC patrols and spawners work as
before on every world.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:36:35 -05:00
Aaron Kimbrell
ee9c6288cb fix(zone): a scene transition links the scenes of its first and last points
Scene transitions of versions 34-38 have five points. The client links
the scenes of the first and the last one (ZoneLoader::ReadZoneFile,
1.10.64: points[0] and points[count - 1], for every version); the scene
graph used the first two, so in those zones it linked a scene with the
one of the transition's second point instead.

Of the 130 five-point transitions on disk, six (all in one older client
zone) have a different scene at the second and last points; the rest
link the same scenes as before.
Live zones are unchanged.

Check in game: nothing to check (no live zone changes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:36:35 -05:00