Commit Graph

74 Commits

Author SHA1 Message Date
Aaron Kimbrell
17425c1e44 fix(zone): read a zone's paths as the chunk the client reads
The client reads a zone file's paths as a chunk of its own
(LuzFile::ReadLUZFile copies the u32-sized chunk, ReadLUZPaths reads
it, 1.10.64) and drops every path when it refuses any of it:
- a chunk version of 2 or more, or 10000 paths or more (ReadLUZPaths);
- a path of version 19 or more, 10000 waypoints or more, or a
  waypoint with more than 99 name/value pairs (LevelPath::FromBuffer).

The reader read the paths straight from the file, trusting the chunk to
be well formed. It now reads the chunk by its size, reads the paths
from it with the client's limits, and when one is refused the zone has
no paths (logged), while the rest of the zone file still reads.

Every zone file on disk reads the same paths as before.

The unit tests' sample zones now give the path chunk its real length.

Check in game: moving platforms, NPC patrols and spawners work as
before on every world.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:36:35 -05:00
Aaron Kimbrell
ee9c6288cb fix(zone): a scene transition links the scenes of its first and last points
Scene transitions of versions 34-38 have five points. The client links
the scenes of the first and the last one (ZoneLoader::ReadZoneFile,
1.10.64: points[0] and points[count - 1], for every version); the scene
graph used the first two, so in those zones it linked a scene with the
one of the transition's second point instead.

Of the 130 five-point transitions on disk, six (all in one older client
zone) have a different scene at the second and last points; the rest
link the same scenes as before.
Live zones are unchanged.

Check in game: nothing to check (no live zone changes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:36:35 -05:00
Aaron Kimbrell
6a8e8ece04 fix(zone): keep the scenes of a zone file the client loads
LuzReader::ReadScenes (1.10.64) keeps a zone's scenes by scene ID and
layer, so a later scene with the same ID and layer replaces an earlier
one. ZoneLoader::ReadZoneFile then goes through them in scene ID order
and
- leaves out every layer of a scene ID that has no layer 0 (General)
  scene;
- before version 41 loads only the layer 0 scene of each ID; from 41 on
  the other layers (Audio, FX) are loaded with it.

ZoneFile now keeps the scenes the same way, so the world loads the same
scenes as the client. No zone file on disk has a duplicate, a scene
without a General layer, or a non-General layer before version 41, and
their scenes are already in ID order: what every file reads is
unchanged.

Check in game: nothing to check (no live zone changes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:36:35 -05:00
Aaron Kimbrell
9f390958f5 fix(zone): scenes of version 30-32 zone files are numbered like the client
Zone files before version 33 have no scene IDs. The client gives each
scene its index in the file as its ID, -1 past 255, with layer 0
(LuzReader::ReadScenes, 1.10.64, via the LWOSceneID from the loop
index); the reader left every such scene at ID 0, so a zone with several
scenes had them all collide on one ID.

The version 30 zones on disk have one scene each, so what they read is
unchanged.

Check in game: nothing to check on live worlds (all are version 36+).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:36:35 -05:00
Aaron Kimbrell
d4fc595e2d feat(zone): read zone files of every version the client reads
Zone files older than version 30 were refused. The client reads them
(1.10.64):
- LuzFile::ReadLUZFile reads a version below 20 as 20;
- LuzReader::ReadScenes reads a scene of a version 20-29 file as a bare
  u32 SceneTable ID, with no file name;
- ZoneLoader::ReadZoneFile takes those scenes in SceneTable ID order,
  looks each one up in the CDClient SceneTable and, when there is a
  row, makes it a scene with the next index (0, 1, ...; -1 past 255) as
  its ID and the row's sceneName as its file; a scene with no row is
  left out.

ZoneFile now reads these files (FileFormatVersion::Oldest = 20 for
anything older), keeps the SceneTable ID on ZoneScene::sceneTableID,
and ZoneFile::ResolveSceneTable does the lookup the client does; the
world looks the names up in the CDClient SceneTable.

The only such file on disk (version 20) now reads.

Check in game: nothing to check on live worlds (all are version 36+).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:36:35 -05:00
Aaron Kimbrell
34ece7f640 test(zone): damaged zone files are skipped by the MD5 of their bytes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 06:21:06 -05:00
Aaron Kimbrell
d5563ad5da refactor(zone): LWOSCENEID lives with the zone code, its layer is an eSceneType
LWOSCENEID and LWOSCENEID_INVALID move from dCommonVars.h to
dZoneManager/LWOSCENEID.h; only the zone code uses them. The layer half
is the scene's layer, now the new eSceneType (dCommon/dEnums) instead of
a uint32_t: General (0), Audio (1, the *_audio.lvl scenes named
"Audio") and FX (2, Nexus Tower's *_fxs.lvl scenes named "FXs"), the
three layers the zone files of every client on disk use. ZoneScene's
sceneType is an eSceneType too, so a scene's LWOSCENEID is built from it
directly, and the dashboard compares against eSceneType::General instead
of 0. The zone checksum still hashes the layer's number.

Check in game: every world loads (the zone checksum the client checks is
unchanged), Nexus Tower included.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 04:50:18 -05:00
Aaron Kimbrell
0143ed834c test(zone): read every zone and scene file of the clients on disk
ClientZoneFilesTests reads every .luz and .lvl under DLU_CLIENTS_DIR
(default ~/Documents/luclients) and expects each to read in full; the
tests are skipped when no client is there. Left out: empty files, sd0
files (the loose sd0 files of the 1.7.45 and 1.9.76 clients are all cut
up after their second chunk; the server gets its files uncompressed from
the packs) and one damaged scene file, which claims 81 objects
in 910 bytes.

With the zone and scene reader fixes before this every file of the
0.179.12, 1.7.45, 1.9.76, 1.10.64 and unpacked clients
reads.

Check in game: nothing (test only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 04:50:18 -05:00
Aaron Kimbrell
59e1d8f7c1 fix(level): read pre-chunk scene files of versions 31 and 33
Two fields of the scene files from before chunks were read for versions
the client does not read them for (SceneLoader::ReadLvlFile, 1.10.64):
- the "important" byte after the version and type is there only from
  version 32 (SceneLoader::ReadLvlHeader, 0x010117d0: headerVersion < 32 sets it to 0);
- the five strings after the skydome's file are there only from version
  34 (SceneLoader::ReadSkydomeInfo, 0x0102f550: headerVersion > 33); the reader took
  them from version 33.

Seven scenes of the alpha leak (versions 31 and 33: Gnarled_assetsMIKET,
dennis_gnarled_forest_02/_03-ninja/_03-pirate_0, scale_TEST_2,
scale_test_2_0, scale_test_general) now read with all their objects.
Every other scene file on disk reads the same objects as before.

Check in game: every world spawns its objects as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 04:50:18 -05:00
Aaron Kimbrell
e7ec11759e fix(zone): spawner paths have an activate-on-load byte only from version 9
The client reads a spawner path's activateOnLoad byte only when the
path is newer than version 8 (LevelPath::FromBuffer, 1.10.64:
pathVersion > 8) and otherwise keeps its default of 1
(Path::InitializeSpawnerInfo). The reader read the byte for every
version, which would misread an older spawner path from there on, and
its default was 0. The byte is now read from version 9, and an older
path's spawner is active on load as in the client.

No zone file on disk has a spawner path older than version 9, so what
the world spawns is unchanged.

Check in game: nothing to check (spawners on live worlds are unchanged).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 04:50:18 -05:00
Aaron Kimbrell
0d2fcc7dff feat(zone): read version 1-2 paths
Paths before version 3 are laid out differently (LevelPath::FromBuffer,
1.10.64: pathVersion < 3):
- the type is a u8-length wide string, "platform" for a moving platform
  and anything else ("npc") for a movement path, instead of a u32;
- the flags and behavior u32s follow as in later versions;
- every waypoint, whatever the path's type, has a rotation, lock player
  byte, speed and wait time and then its name/value pairs.

The reader treated the type name's first bytes as the type and misread
everything after it. The LUP group zones of the 0.179.12
client (lup_group_1b, _2a, _6a) now read with all their paths; one of
them read before with a garbage type on a waypoint-less path. Every
other zone reads the same as before (no live zone has a path older than
version 3).

Check in game: nothing to check on live worlds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 04:50:18 -05:00
Aaron Kimbrell
942d0ff8ad fix(zone): PrePreAlpha zone files have no zone name or description
The client reads a zone's name and description only when the file is
newer than version 30 (LuzFile::ReadLUZFile, 1.10.64: revision > 0x1e);
a version 30 file ends at its terrain file's name. The reader read both
strings for every version, so version 30 files ran off their end.

The four version 30 zones of the alpha leak (scale_TEST_ASSETS,
Gnarled_assetsMIKET, dennis_gnarled_forest_02/_03-ninja) now read. Some
of them have a stray name string after the terrain file's name, which
the client does not read either. Every newer file reads the same as
before.

Check in game: nothing to check on live worlds (none is version 30).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 04:50:18 -05:00
Aaron Kimbrell
69093c6816 fix(zone): zone files older than version 30 are refused
Before version 30 (PrePreAlpha) a zone file's scene is only a u32 scene
ID, with no scene file name (LuzReader::ReadScenes, 1.10.64:
revision < 30 reads the ID alone; LuzFile::ReadLUZFile treats anything
below 20 as 20). The reader read a file name and then an ID for those
versions, which is not the format, and the world could not load such a
zone's scenes anyway. Reading one now throws a runtime_error that says
why, before anything else is read.

No zone file of any client on disk is older than 30 except an unnamed
editor stub (a res/.luz, version 20).

Check in game: every world loads as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 04:50:17 -05:00
Aaron Kimbrell
04852fce7e fix(zone): zone files of version 37 have a u32 scene count
The scene count is a u8 before version 37 (LateAlpha) and a u32 from 37
on (LuzFile::ReadLUZFile, 1.10.64: revision < 37 reads a byte). The
reader took the u8 for 37 too, so version 37 files misread from their
scenes on and failed.

No live zone is version 37; four older LUP zones
now read in full.
Every other zone file on disk reads the same as before.

Check in game: nothing to check on live worlds (none is version 37).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 04:50:17 -05:00
Aaron Kimbrell
97a1453bf7 feat(zone): read the zone boundary lines of .luz files
Between the scenes and the terrain file's name a zone file has its
boundary lines: a u8 count, then per line a normal, a point, the
destination zone, the destination scene ID and a spawn location. The
reader took that spot for a u8-length "zone path" string, which is the
same bytes only when there are no boundaries; a zone with boundaries
would have misread everything after it.

The destination is one u32 in the client (LuzReader::ReadZoneBoundaryLines,
0x01018490 in 1.10.64: map ID in bits 0-15, instance ID in bits 16-31,
clone 0), read here as two u16s into an LWOZONEID with clone 0. The
boundaries are kept on ZoneFile::zoneBoundaries.

No zone of the 1.10.64 client (or any other client on disk) has
boundary lines, so what is read from them is unchanged.

Check in game: every world loads and zone transfers (launch pads,
rockets, portals) work as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 04:50:17 -05:00
Aaron Kimbrell
e9ff5f402d feat(world): read UgcDownloadFailed and log real download failures
World packet 120 (UgcDownloadFailed: resType u32, blueprint ID, status
u32, character ID; lu_packets, 31 bytes after the 0x53 as the client
sends it) was logged as an unknown world packet. The client sends it
from LWOResMgr2Interface::FinishResourceRequest (0x0105e5c0) for every
blueprint file (player model, car or rocket build) whose request did not
end with HTTP 200, including files it did not download at all (status
0). Live: 1525 packets, 1520 with status 0 (all four file types, around
load and FetchModelMetadataRequest), 5 with 404. Live sent nothing back
and the sessions carried on.

The logout the client does on failed UGC downloads
(NET_DISCONNECT_FAILED_DOWNLOAD_UGC, MainThread_LogoutDueToConnectionFailures
0x0102b9c0) is its own decision after repeated connection failures to the
UGC HTTP server; there is no server message that prevents or triggers it.
So the server only records it: a log line for an HTTP failure (for
finding missing files on the UGC server), a debug line for status 0.

MessageType::World gains UGC_DOWNLOAD_FAILED = 120 (appended; the magic
enum range goes to 120).

Check in game: nothing changes for the player. With a property that has
models, load it: the world server log has no "Unknown world packet 120"
lines; if a model file is missing on the UGC server there is one "failed
to download blueprint" line naming it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 03:44:06 -05:00
Aaron Kimbrell
8d6fe4b116 fix(ghosting): scene ghosting follows the client's scene streaming
Re-checked against client 1.10.64: Zone::Run calls StreamScenesAroundPosition
with the ghost reference position, and with the controlled object's own
position only when the two differ (ghost reference override on). The client
loads the scene under the reference point and the global scene; with the
override on it also loads the scene under the player and its connected
scenes. The cell lookup (floor(v + 0.5), x cell * resolution + z cell), the
transition pairs and FixupInvalidTransitions match what ZoneScenes does.

Scene ghosting now adds the scenes around the player while the ghost
reference is overridden (cinematics), and the comments say the server keeps
each scene's neighbours too (a superset, so objects across a transition
exist before the player crosses it).

Check in game (with ghosting_scenes=1): walk across scene transitions in
Avant Gardens and Gnarled Forest; objects on both sides show up and nothing
pops in at the line. Play a cinematic that moves the camera away (e.g. a
mission cinematic) and objects around the player stay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 02:42:02 -05:00
Aaron Kimbrell
a52e777097 fix(messages): client names for game messages 792, 793, 915 and 916
The 1.10.64 client's message table (GameMessage::<Name>::Initialize) names
792 DeletePropertyResponse, 793 CreateModelFromClient, 915
PropertyModerationAction and 916 PropertyModerationActionResponse.
Only the enum names change; the IDs stay pinned.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 23:44:44 -05:00
Aaron Kimbrell
b5e92ce97c fix(messages): swap the names of game messages 1593 and 1594
The client registers SetPropertyModerationStatus as 1594
(GameMessage::SetPropertyModerationStatus sets id 0x63a; no client
code uses 1593). The enum had the two names one slot off. Names only;
the IDs are unchanged and the pin tests still check both values.

In game: nothing changes; neither message is sent yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:33:36 -05:00
Aaron Kimbrell
109a936798 feat: live updates move every server onto a new build without a restart
With new binaries in place, master moves everything onto them while the
server keeps running (the dashboard's Live update, /liveupdate or SIGUSR2 to
master):

- database migrations of the new build first; a failure stops there
- UGC finishes the jobs it is running (queued rows stay pending), auth
  restarts, chat hands its teams to master for the next chat server; master
  starts the new processes and retries ones that don't come back
- once the new chat server is up (CHAT_SERVER_READY) every world connects at
  once and sends its players again (LoginSessionNotify resync, no login logged)
- every world instance is replaced with an instance migration: public worlds
  and private ones (same password) at once, properties after the old instance
  saved and froze the property (MIGRATE_PREPARE: no building, claiming or
  saving there any more), activity zones and character select once their
  players left or after a wait; empty instances just stop, zones in
  prestart_worlds get a new one first
- the dashboard restarts last and picks the status up again

Players land where they stood (position carried in CarriedPlayerState, also on
properties and Moon Base). Draining instances get no new players
(InstanceMigration::AcceptsNewPlayers) and show as "Moving players" in the
world list. The order lives in LiveUpdateMachine.h without master state and is
unit tested; master's glue is LiveUpdateCoordinator. Master itself is not
replaced. Message IDs are appended only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:23 -05:00
Aaron Kimbrell
17513a8390 fix: splitting and normalizing models moves every bone and rigid, in file order
Lxfml::NormalizePosition moved only the first bone of each part, and never
the rigid systems, so after a save a flexible part's other bones and every
Rigid kept their world positions while the rest of the model was moved to
its own origin. Now every Bone and every Rigid moves by the same amount, and
the box that places the model holds every bone. Split maps every bone of a
part too, so a rigid system holding only a later bone keeps its brick.

Also:
- a model with no readable bone is kept as it is at the origin instead of
  getting a center 10000 up
- the math is done in doubles and numbers are written as the shortest text
  that reads back as the same float (-0.4, not -0.400002; 12.1678, not
  12.1677)
- bricks and rigid systems come out in the file's order, so the same model
  always splits into the same bytes
- parts over 5 MB are normalized like any other (the input is capped at
  10 MB), and the loop's safety limit no longer drops every later model

The pivot is still snapped to the 0.8 grid (the bricks don't move in the
world; the model's position stays on the grid), which can put it half a
stud from the middle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:19 -05:00
Aaron Kimbrell
2da96ada83 feat(dashboard): 3D views show a zone's scenes as the game loads them
A "Scenes" choice in the world and property 3D views: every scene (as
before), the scenes the game keeps loaded around the camera or the followed
player (the scene under it from the terrain's scene map, the scenes its
transitions connect to, and the global scene, following as it moves), or
scenes picked from a list (world view). The lighting blends to the lighting
of the scene under the focus, as the client blends between scenes.

The scenery manifest now carries each object's scene, the zone's scenes with
their neighbours and lighting, and the scene map as runs (37 KB for Avant
Gardens); scenery-core.js finds the scene at a point exactly as ZoneScenes
does (checked against it on 2000 points of Avant Gardens).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:17 -05:00
Aaron Kimbrell
83b87744c1 feat: scene ghosting, as the client streams a zone's scenes
The client keeps the scene under the player loaded, the scenes the zone
file's transitions connect to it, and the global scene
(Zone::StreamScenesAroundPosition 0x0108a3f0, TerrainManager::GetSceneAtPos
0x01069010, the connected scenes at 0x01066500; transitions naming a
missing scene dropped as Zone::FixupInvalidTransitions 0x010842e0 does).

- ZoneScenes (dCommon): the terrain's scene map lookup and the scene graph,
  shared by the world server and the dashboard.
- Objects remember the scene they were placed in (spawners pass theirs on).
- ghosting_scenes=1 (world config, off by default): players get the objects
  of their loaded scenes instead of the ones within the ghosting distances;
  objects from no scene go by the scene under them. Zones without a scene
  map keep distance ghosting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:17 -05:00
Aaron Kimbrell
c939a69240 feat(dashboard): UGC list shows a model's given name, and Took/CPU/RAM columns
The models list's Took, CPU and RAM (est.) are separate columns, each
sortable (sort=slowest|cpu|memory). The File column shows the name a
player gave the model where it is placed, with the upload's extension
(the upload's file name is its tooltip), and the name sort uses it
(case-insensitive, so SQLite and MySQL agree).

The config layer test no longer assumes the build's sharedconfig.ini has
no mysql_host; it checks that the database-supplied value isn't used.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:16 -05:00
Aaron Kimbrell
37459e0019 fix(properties): top property slots show only their own world
The news screen's slot tooltip names the slot's own property world
whatever property is sent for it, so a property of another world was
shown under the wrong name. Each slot now only shows a property of its
own world: a location stored by the older per-slot setting is kept in
the table but not used, full auto fills each slot with its own world's
top property instead of the top four across every world, and the
dashboard no longer offers a location or candidates from other worlds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:14 -05:00
Aaron Kimbrell
ceec638e52 feat(common): Sd0::Compress, a default Sd0 and room for chunks that don't shrink
Sd0 gets a default constructor and Sd0::Compress(data), the raw sd0 bytes of
some data (what the client reads for UGC files it downloads without 3D
services). FromData compresses into a heap buffer big enough for a chunk that
grows when deflated (it used a 256 KiB stack buffer, so random data failed and
workers carried a large stack frame), and drops a half-written result.

Tests: chunks as the 1.10.64 client inflates them, incompressible data, empty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:13 -05:00
Aaron Kimbrell
8ab7e496b0 feat(bbb): answer FetchModelMetadataRequest for brick built models
The client asks for a model's metadata (name, owner, behaviors and its blueprint's
bricks and box) when it shows a brick built model item's tooltip, a model on a
property or an exhibit, and shows BBB_LOADING_BLUEPRINT until it gets it. The
world server never answered. It now does as live did: UG data for the model
(found among the player's items by subkey, else among placed models) and, for a
brick built model, the blueprint data from its ugc row and LXFML.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:10 -05:00
Aaron Kimbrell
13ad679df1 feat(net): count every server's packets and send a traffic report every 5 seconds
TrafficStats keeps packets and bytes in and out per second and the busiest
packet and game message types. dServer counts at its send and receive calls
and adds RakNet's connection statistics (datagrams, resends, ping); the report
goes to master as SERVER_TRAFFIC (appended), which passes it to the dashboard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:31:01 -05:00
Aaron Kimbrell
abf1cc1d80 feat: UGC server that makes and serves player models' meshes and icons
A new server (dUgcServer, started by master with enable_ugc_server=1) that
takes unprocessed ugc and ugc_modular_build rows from the database and makes
what the client downloads with UGCUSE3DSERVICES: an optimized NIF (hidden
faces removed, ambient occlusion baked into vertex colors) and a 128px DDS
icon for player models, rendered by a software rasterizer from the client's
LDD brick primitives, and icons for cars and rockets assembled from their
modules per ModularBuildComponent/ModuleComponent. It serves them, with the
models' LXFML, over HTTP in the client's UGCC<dc>/3DOPTIMIZED and
IMAGE128DDS layout with .gz and .checksum files, and keeps its folder under
a size cap.

Processing state lives in ugc.is_optimized plus new processed_at,
process_attempts and process_error columns (and the same on
ugc_modular_build). ServiceType::UGC is appended. NifFile moves to dCommon
and records named node transforms for the modules' attach points.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:57 -05:00
Aaron Kimbrell
542f0a89f0 refactor: remove the packet macros, WriteHeader and PacketUtils
Nothing in the server writes or reads a packet by hand any more, so the
helpers for doing so go:
- CBITSTREAM, CMSGHEADER, CINSTREAM, CINSTREAM_SKIP_HEADER, SEND_PACKET,
  SEND_PACKET_BROADCAST and HEADER_SIZE leave dCommonVars.h;
- the free BitStreamUtils::WriteHeader (LUBitStream::WriteHeader writes the
  same bytes) and the unused PacketUtils::SavePacket are deleted.
The last raw reads are replaced: WorldServer builds its input stream
directly, the master packet logs read the header with
LUBitStream::ReadHeader instead of peeking at packet->data[1] and [3], and
MessageInspector reads a sent game message's header with the new
NetGameMsg::ReadPacketHeader (the counterpart of WritePacket) instead of
memcmp/memcpy. packet->data[0] is still compared with RakNet's own
connection IDs.

The frozen oracles keep using the macros verbatim through the test-only
tests/dGameTests/LegacyPacketMacros.h; the HeaderSkip tests, which only
tested CINSTREAM_SKIP_HEADER, are removed.

docs/PacketArchitecture.md: "where we are" now describes the final state
and what still touches raw bytes (RakNet IDs, replica headers, behavior bit
streams), and a new section collects the known wire discrepancies found
during the conversion, with client addresses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:56 -05:00
Aaron Kimbrell
b2a1e9e0b8 refactor: remaining game messages as structs, switch removed
Every game message still written or read by hand is now a NetGameMsg with
Serialize and Deserialize, in per-domain files:
- MovementMessages: teleport, platforms (resync and its request), orient to
  angle, node rotation lock, gravity scale, jetpack mode, control scheme,
  respawn checkpoint, rails (set/start, ready, cancel, arrived), mount
  inventory ID, dismount complete, possession ack, ghost reference override
  and position, camera cycling (eCameraTargetCyclingMode moves here).
- ZoneMessages: player loaded (the old PLAYER_LOADED case), ready for
  updates, player ready, restore to post load stats, server done loading,
  invalid zone transfer list, zone summary display/dismissed, level
  processing complete, object world state, and the localized announcement
  WorldMigration wrote by hand.
- PlayerMessages: chat mode, GM level, LEGO score, currency, reputation,
  GM invis, pickup currency, zone and player statistics, chat commands, bug
  reports, verify ack.
- ObjectMessages: fire event client/server side, notify client
  (zone) object, notify object, script network vars, failed preconditions,
  terminate interaction, set name, request use, request server object info.
- QuickBuildMessages: notify state, enable, cancel.
- ActivityMessages gains match response/update/request, leaderboard request
  and data, shooting gallery score/rotation/fire, activity state change.
- MissionMessages gains MissionDialogueCancelled (a no-op, as before).
The wire structs left in GameMessages.h move to their domains (tooltip and
emote to Effects, loot and item use to Inventory, model build to Building,
behavior sound to Property, skill sets to Skill) and gain the missing
direction. The dismount logic moves to PossessorComponent::OnDismountComplete.

Every inbound message is registered in the GameMessageHandler map; the
switch is gone, and GameMessages.cpp only holds the GameMsg/NetGameMsg base
code. Call sites build the structs (NotifyClientObject, TerminateInteraction,
Teleport, PlatformResync, FireEventClientSide, NotifyObject and
NotifyClientZoneObject get convenience constructors like PlayFXEffect).
Dead senders are dropped: SendSetShootingGalleryParams (no callers, field
order was a guess), SendTeamPickupItem (the struct already existed),
SendRequestActivitySummaryLeaderboardData (the struct covers it).

Verified with RemainingMessagesTests: every old Send* function is frozen
verbatim in Legacy/RemainingMessagesLegacy.h and compared byte for byte
(same bits, destination and broadcast flag) over grids of inputs; every old
Handle* read sequence is frozen as a Read* oracle and compared with the
struct's Deserialize; round trips, truncation and a golden packet.
PlayerLoaded (0x00dc36f0), SetGMLevel (0x00dd6230), MissionDialogueCancelled
(0x00d9cc10) and LocalizedAnnouncementServerToSingleClient (0x00f23c50)
were checked against the client. Behaviour notes: an inbound message that
fails to deserialize is dropped, so ParseChatMessage over MAX_MESSAGE_LENGTH
is dropped instead of truncated, and PLAYER_LOADED / READY_FOR_UPDATES /
MISSION_DIALOGUE_CANCELLED now read their (unused) client fields.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:56 -05:00
Aaron Kimbrell
e36f894f1f feat: bind_ip setting for the server sockets
bind_ip in sharedconfig.ini picks the local IPv4 address every server's
RakNet sockets listen on (auth, chat, master, world, the dashboard's
connection to master), separate from external_ip, the address players
are sent. Empty or 0.0.0.0 keeps listening on all interfaces; localhost
means 127.0.0.1. Anything that isn't an IPv4 address stops the server
with an error, and each server logs what it bound to.

Fixes #225

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:54 -05:00
Aaron Kimbrell
d95eab1212 feat: property reputation from visitors, resistant to farming
Property worlds now give their property reputation for the time other people spend on it, which
fills the property lists and the news screen's Today's Top Properties. Visitors are counted per
account; the owner's account, accounts linked to it and staff don't count. A visit earns nothing
for the first property_reputation_min_visit seconds (WorldConfig's propertyReputationDelay), then
each active minute (the visitor moved) earns reputationPerMinute times a multiplier, for a capped
number of minutes per visit. Repeat visitors earn less the more recent days they already gave
reputation, and each visitor and each property have a daily cap. Every parameter is a setting;
what each account gave each property per day is kept in property_reputation_visits. The rules are
pure functions with unit tests.

Fixes #636
Fixes #637

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:52 -05:00
Aaron Kimbrell
680615ba9f feat: optional property rent
Off unless property_rent_enabled is on. Each property world's rent comes from its PropertyTemplate
row (minimumPrice every rentDuration x durationType; Block Yard is free), unless the new Property
Rent dashboard page sets another price or period (property_rent_manage). Rent is taken from the
owner's coins shortly after their character loads, with a mail receipt; unpaid rent is mailed and,
after property_rent_grace_days, makes the property private until it is paid, like live. The
property management component refuses public or best friends privacy while rent is overdue and a
property world that loads overdue makes itself private. Property game messages are unchanged.

Fixes #943

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:52 -05:00
Aaron Kimbrell
5a9a3e380b refactor: master packets as structs
Every MASTER service packet is now an LUBitStream struct (docs/PacketArchitecture.md,
PR 14) and the master server's switch is a dispatch map (PacketDispatcher), as are the
master handlers of the world, chat and dashboard servers.

- dNet/MasterPackets.h: RequestZoneTransfer, RequestZoneTransferResponse, ServerInfo,
  RequestSessionKey, SetSessionKey, SessionKeyResponse, NewSessionAlert, PlayerAdded /
  PlayerRemoved, CreatePrivateZone, RequestPrivateZone (passwords still cut to 50
  characters when read), WorldReady, WorldReadyInfo (WORLD_READY to the dashboard),
  PrepZone, Shutdown, ShutdownResponse, WorldShutDown (SHUTDOWN_RESPONSE to the
  dashboard), ShutdownUniverse, AffirmTransferRequest/Response, RequestServerList,
  ServerListResponse, DashboardShutdown, ConfigReload, InstanceShutdown. The Send*
  functions are gone; MasterPackets::SendToMaster(msg) and SendTo(sysAddr, msg) send a
  struct.
- The dashboard and instance migration structs (PlayerAction, DataChanged, Dashboard
  messages, MessageCapture, InstanceMigration) are LUBitStreams of the MASTER service now
  and moved to dNet/master/, included by MasterPackets.h. Their payloads are unchanged;
  master forwards them by re-serializing the struct instead of copying raw bytes.
- InstanceManager, ZoneInstanceManager, MigrationCoordinator, dServer (server info, zone
  transfer response), auth (SET_SESSION_KEY), the world (session keys, player added and
  removed, world ready, shutdown response, affirmations, prep zone, shutdown universe) and
  the dashboard (server list, instance shutdown, config reload, announcements, player
  actions, message capture) send and read structs.
- The login stamps are a `stamps` field of RequestZoneTransfer and
  RequestZoneTransferResponse (read leniently as before: a message without them reads as
  empty); master adds its stamps in the REQUEST_ZONE_TRANSFER handler and when it answers,
  as it did.
- InstanceManager::GetInstanceBySysAddr takes a const address.

Verified: tests/dGameTests/dNetTests/Legacy/MasterPacketsLegacy.h is a verbatim copy of
the old writers and readers; MasterPacketsTests requires identical bytes for a grid of
inputs, checks the old readers read what the structs write, round trips and truncation,
hand written golden packets, that zone transfers without stamps still read, that the dashboard/migration structs write what
"header + Serialize" wrote, and that the dispatcher drops truncated packets. No wire
bytes changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:50 -05:00
Aaron Kimbrell
040d7ec067 fix: ignore whitespace around config keys and values
"client_location = ../client " or a value with a trailing space is a
common setup mistake that surfaces much later as an unrelated error (a
missing client folder, a failed database login). Keys and values are now
trimmed of spaces, tabs and line endings when the ini is read, which
also covers the \r of files saved with Windows line endings. Spaces
inside a value are kept, and lines with an empty key are ignored.

Verified with a new unit test that loads an ini with padded keys and
values.

Refs #1113

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:45 -05:00
Aaron Kimbrell
8a2ccb1ae7 fix: bound AMF3 decoding and stop hashing client keys
Client-sent AMF (CONTROL_BEHAVIORS) was decoded without limits on the
associative part of arrays, on nesting depth, or on the total number of
values. Associative entries went into an unordered_map with the standard
unseeded string hash, so a client could pick colliding keys and make
insertion quadratic, and deeply nested arrays recursed until the stack
overflowed, crashing the world server.

- The associative map is now an ordered std::map (O(log n) whatever the
  keys, deterministic serialization order).
- Each array allows at most 10,000 associative entries, the same as the
  existing dense limit, which is now checked before anything is read.
- Arrays may nest at most 32 deep and one deserializer reads at most
  100,000 values. Every limit throws, which the only caller already
  catches and drops the message.
- Inserting a duplicate key keeps the last value and no longer returns a
  reference to a value that was destroyed when the key already held null.

Verified with new unit tests for each limit (including a 100,000 deep
nesting that previously overflowed the stack) and the existing live
packet test, which still decodes.

Fixes #2035

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:44 -05:00
Aaron Kimbrell
e213a7aeff feat: web dashboard and playground work
The NexusDashboard-parity dashboard (dDashboardServer) and everything built on it on the experimental branch:
accounts, characters, properties and moderation tools, permissions shared with in-game slash commands, economy
reports, World 3D and property 3D views with client scenery, scheduled events (features, vanity changes, live
events, announcements, restarts), vanity files and events, the CDClient browser, the message inspector with saved
captures, chat filter tools, community challenges, live ops, the AI moderator helper, and the server-side changes
they need.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:43 -05:00
Aaron Kimbrell
b1930ed4ed feat: instance migration messages and planning
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:36 -05:00
Aaron Kimbrell
350d35dab5 chore: foundations for struct based packets and game messages
First step of moving hand written bitstream code to struct based
messages (see docs/PacketArchitecture.md). No wire changes.

- GameMsg is now only a server-internal event (delivered to handlers
  registered with RegisterMsg); NetGameMsg is a wire message with
  Send(sysAddr) (UNASSIGNED broadcasts), SendToClient(sysAddr) (one
  client, never broadcasts), WritePacket, Serialize, Deserialize and
  Handle. Mixing them up is now a compile error. NetGameMsgEvent<T> /
  DeliverLocally carry a wire message to local handlers; loot drops,
  pickup, the object debugger, GM invisibility and model RequestUse
  use them. The GM invisibility message keeps its target, so its bytes
  are unchanged.
- Behaviour change: GameMessageHandler logs and drops messages whose
  Deserialize fails instead of handling them with default fields (the
  4 messages already registered: RequestUse, RequestServerObjectInfo,
  ShootingGalleryFire, PickupItem).
- LUBitStream (kept as on main) gets a virtual destructor (Mail deleted
  derived packets through the base) and WritePacket, also used by
  ChatPackets::SendRoutedMsg with identical bytes.
- BitStreamUtils::WriteOptional/ReadOptional for default-flag fields
  and WriteLengthPrefixed/ReadLengthPrefixed for length prefixed
  strings.
- Every message ID enumerator (MessageType::*, ServiceType, Mail's
  wire enums) is pinned with static_asserts, so renumbering or removing
  one fails to compile.
- Tests: dServerMock copies each sent packet (it kept a pointer to the
  caller's destroyed BitStream); PacketTestUtils.h compares packets bit
  for bit; helper tests use hand computed golden bytes and equality
  with the hand written patterns they replace; compile-time checks
  keep the wire/internal split in place.
- docs/PacketArchitecture.md: survey, target architecture,
  conventions, verification method and migration plan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:30:35 -05:00
David Markowitz
a156a8fcba fix: security vulnerabilities (#1980)
* fix: security vulnerabilities

Tested that all functions related to the touched files work

will test sqlite on a CI build

* fix failing test

* ai feedback

* add buffer size checking

* use c_str

* dont log session key

* Try this for a mac definition

* be quiet apple
2026-06-07 20:59:11 -07:00
Aaron Kimbrell
c2dba31f70 fix: bbb splitting dupe issue (#1908)
* fix bbb group splitting issues

* address feedback
2025-10-15 16:45:09 -07:00
Aaron Kimbrell
ce28834dce feat: lxfml splitting for bbb (#1877)
* LXFML SPLITTING
Included test file

* move base to global namespace

* wip need to test

* update last fixes

* update world sending bbb to be more efficient

* Address feedback form Emo in doscord

* Make LXFML class for robust and add more tests to edge cases and malformed data

* get rid of the string copy and make the deep clone have a recursive limit

* cleanup tests

* fix test file locations

* fix file path

* KISS

* add cmakelists

* fix typos

* NL @ EOF

* tabs and split out to func

* naming standard
2025-10-10 23:07:16 -05:00
David Markowitz
e4c2eecbc7 add msg handling (#1737) 2025-01-20 00:42:15 -06:00
David Markowitz
8b56b0b7ba fix: use current binary directory mariadb shared object and dont override env variable (#1669)
* mac stuff

grab correct mariadb file and dont override env variable

fix for unix

Update CMakeLists.txt

unix only

* get that dylib
2024-12-08 00:36:49 -06:00
jadebenn
53877a0bc3 refactor: Rewrite AMF and property behavior logic to use smart pointers, references, and string_views over raw pointers and std::string& (#1452)
* Rewrite AMF and behavior logic to use smart pointers, references, and string_views over raw pointers and std::string&

* fix m_BehaviorID initialization

* Fix BlockDefinition member naming

* remove redundant reset()s

* Replace UB forward template declarations with header include

* remove unneeded comment

* remove non-const ref getters

* simplify default behavior id initialization

* Fix invalidated use of Getter to set a value

* Update AddStripMessage.cpp - change push_back to emplace_back

* fix pointer to ref conversion mistake (should not have directly grabbed from the other branch commit)

* deref

* VERY experimental testing of forward declaration of templates - probably will revert

* Revert changes (as expected)

* Update BlockDefinition.h - remove extraneous semicolons

* Update BlockDefinition.h - remove linebreak

* Update Amf3.h member naming scheme

* fix duplicated code

* const iterators

* const pointers

* reviving this branch

* update read switch cases
2024-11-18 20:45:24 -06:00
jadebenn
c7dd8205a4 feat: Make use of CMake presets to enable easy switching between debug and release configurations on all platforms (#1439)
* Add MSVC optimization flags

* test moving flags to json

* Update CMakePresets.json

* testing

* trying more variations on the flags

* third test

* testing if these even have any effect

* ditto

* final(?) try for now

* ONE MORE TIME

* trying 'init' flags instead

* export the compile commands so I can see if they're having any effect

* move out g++ O2 flag

* add Linux debug preset

* update CMake presets

* edit macos presets

* try adding build types back to mac

* macos refuses to work :(

* try using compiler flags for mac instead

* fix typo in windows preset

* build reorganization and experimental clang support

* temporarily remove macos build for testing purposes

* updated cmake workflows

* unexclude toolchain dir

* update .gitignore

* fix build directory issue

* edit build script

* update cmake configs

* attempted docker fix

* try zero-initializinng this struct to solve docker issue

* try fixing macos build

* one last MacOS try for the night

* try disabling an apple-specific build rule

* more fiddling with mac test builds

* try and narrow down the macos build failure cause

* try stripping out all the custom macos test logic again

* I'm really just throwing everything to the wall and seeing what sticks

* more macos tinkering

* implib

* try manual link directory specification

* save me

* aaaaaaaaa

* paths paths paths

* Revert "paths paths paths"

This reverts commit 9a7d86aa6c.

* Revert "aaaaaaaaa"

This reverts commit 338279c396.

* Revert "save me"

This reverts commit bd73aa21a9.

* Revert "try manual link directory specification"

This reverts commit 0c2d40632e.

* Revert "implib"

This reverts commit d41349d6ed.

* Revert "more macos tinkering"

This reverts commit 829ec35b57.

* Revert "I'm really just throwing everything to the wall and seeing what sticks"

This reverts commit 1a05b027fe.

* Revert "try stripping out all the custom macos test logic again"

This reverts commit cc15a26ce8.

* Revert "try and narrow down the macos build failure cause"

This reverts commit 5fd86833fa.

* Revert "more fiddling with mac test builds"

This reverts commit 0f843c02c9.

* Revert "try disabling an apple-specific build rule"

This reverts commit 45ec66e976.

* back to debug messages

* see if this re-breaks mac

* are these messages actually somehow fixing the issue?

* was not actually fixed

* add debug messages (again)

* debug try 2

* change runtime output dir

* rename gcc to gnu

* expand cmake presets

* fix preset

* change defaults

* altered cmake configuration scripts

* disable /WX on MSVC

* update github actions

* update build presets

* change gnu and clang build directories to enable consistent artifact generation

* add RelWithDebInfo presets and move -Werror flag into presets.json

* use DLU_CONFIG_DIR envvar

* CMakePresets indentation

* temp fix for MSVC debug builds
2024-11-17 19:03:54 -06:00
jadebenn
84d7c65717 consolidate the messagetype enums into a single namespace (#1647) 2024-11-17 18:39:44 -06:00
jadebenn
fafe2aefad chore: Nitpicking-utils (#1549)
* nit

* GeneralUtils const-correctness and minor fixes

* use copy instead of reference for char iteration loops

* fix typo and reorganize some functions
2024-04-14 23:14:54 -07:00
Aaron Kimbrell
feeac2e041 feat: refactor slash commands system into more scalable system (#1510)
* WIP, but working

* Scaffolding

* testing and making it compile again

* move all commands to functions

* renaming to compile

* fix failing tests

idk how these werent failing before.  Seems to have been magic.

* move commandss into their namespace
make help command useful
fix mac error

TODO: remove the multiple not founds/ rework the structure to split into help and handling

* Just need to fill out the fields, but it's all there templated

* Add all aliases, register missing commands

* All help text

* remove test logs

* improvements

pass through added code for optimizations and cleanup as well as reduce the amount of scoping for readability and maintainability

* Update SlashCommandHandler.cpp

* only save command if it is a GM command

* simplify if checks

* remove broken delimiter

* Update SlashCommandHandler.cpp

* Update SlashCommandHandler.cpp

---------

Co-authored-by: David Markowitz <EmosewaMC@gmail.com>
2024-04-08 15:11:59 -05:00