Several game messages read a 32 bit length and then read that many
characters without checking it, so the loop ran for the whole length
even after the message had ended. The length now has to fit in the
data the message has left. This covers the activity state, cinematic
path, death type, removed item info and inventory group messages and
the shared wide string reader.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
An sd0 buffer could hold any number of chunks, and each chunk grew the
inflated data by a full chunk whatever it held, so the data of a chunk
that was not full was followed by a gap. A buffer now holds at most
4096 chunks and inflates to at most 64 MiB, and every chunk is inflated
right after the one before it.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The AMF3 reader read an array inside an array to any depth and kept
reading values after the data had ended. It now refuses arrays nested
more than 64 deep and throws when the data ends before a value does.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The key came from the shared game random engine, which is seeded from
the start time. It now comes from std::random_device through a small
helper. The key stays a 32 bit number, so the packets do not change.
The master and world servers no longer write the key or its hash to
the log.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- Zone::LoadZoneIntoMemory takes the paths the zone file reader read (lost in the merge of the raw terrain loading)
- the terrain reader is main's unchanged: files from before version 32 keep their colour and texture maps as the file
has them, and the dashboard's terrain tests expect that (bounds by the span between the first and last point too)
- the model component test counts the model block only: the item block in front of it is the ItemComponent's
- scene file tests: a file from before chunks keeps only its header, editor settings are a count of colours, and the
tests that log have a logger
- the settings catalog lists export_terrain_to_obj
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a brick by brick save is split into models, each model was centred on
the middle of its bones' origins, on the lowest origin. A brick's origin is at
a corner of it, not its middle, so a single 2x2 came back offset by half a
brick, and the model's floor was taken from the origins rather than from the
bricks.
Now each model is placed as the live server placed it: at the bottom middle
of the box around its bricks. Each brick's own box comes from the client's
brickdb by its design ID and is turned and moved by its bone (all 8 corners),
and the model's box holds every brick's. Its x and z middle and its lowest y
are the new origin, still snapped to the 0.8 grid on x and z, and every bone
moves by that; rigids and rotations are untouched. A design the brickdb has no
box for counts its bones' origins, as before, so no brick is left out.
Lxfml::NormalizePosition and Split take the boxes by design ID, and the brick
by brick save passes the ones read from the client (read on the first save,
then kept).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BrickBounds reads the <Bounding><AABB/> of every Primitives/<designID>.xml in
the client's res/brickdb.zip (through the asset manager, so packed and
unpacked clients both work), by design ID. A design listed only as another
primitive's alias gets that primitive's box. Get() reads the file the first
time it is called and keeps the boxes for every later call.
The zip is read by a small reader in dCommon (Zip::ForEachEntry) that handles
stored and deflated entries, with a raw inflate added to ZCompression.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lxfml::NormalizePosition moved only the first bone of each part, so after a
save a flexible part's other bones kept their world positions while the rest
of the model was moved to its own origin. Now every Bone moves by the same
amount, and the box that places the model holds every bone. Split maps every
bone of a part too, so a rigid system holding only a later bone keeps its
brick.
Only bones move, as the live server did: the Rigid transformations are sent
back as the client wrote them, and rotations are not touched.
Also:
- a model with no readable bone is kept as it is at the origin instead of
getting a center 10000 up
- the math is done in doubles and numbers are written as the shortest text
that reads back as the same float (-0.4, not -0.400002; 12.1678, not
12.1677)
- bricks and rigid systems come out in the file's order, so the same model
always splits into the same bytes
- parts over 5 MB are normalized like any other (the input is capped at
10 MB), and the loop's safety limit no longer drops every later model
The pivot is still snapped to the 0.8 grid (the bricks don't move in the
world; the model's position stays on the grid), which can put it half a
stud from the middle.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ZoneBoundary's destinationZone and destinationSceneID and the scene's NiColor, so main's scene tools and tests read
the zone this branch's reader loads.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dashboard's copy of the terrain reader in dCommon is gone; main's in dZoneManager reads for everyone, and keeps
what the copy had over it: colour and texture maps from before version 32 become RGBA, cut to the resolution the
client keeps. The scene map (ZoneScenes) and the dCommon tests compile against that reader.
A scene ID takes its layer as the zone file numbers it (0 General, 1 Audio) as well as the enum, so main's scene tools
and tests build; /getscene prints the layer as a number.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Appended message ID 50 (dashboard -> master): the update check's state and a
one-line summary. Master logs it whenever the summary changes, so the first
one after master starts says when an update is out.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The client's word check before sending is a light one; the server is the full filter. In normal (whitelist) chat, block list phrases are now stopped even when each word is allowed, and allowed phrases (from the dashboard or a line with spaces in the allowed words file) let their words through together. The dashboard can allow phrases and its message test explains both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shipped list's hashes are 64-bit FNV-1a (checked against a sample of
its words), so it is written again as version 3: same hashes, duplicates
dropped, sorted. Nothing else changes; it can be replaced by building
from blocklist.txt. issue 215
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The filter stored and compared words by std::hash<std::string> in size_t,
which differs between standard libraries and platforms, so a .dcf made on
one system never matched on another and the block list never worked
there (issue 215).
- ChatFilterCore.h: 64-bit FNV-1a over the entry's bytes, ASCII lower
case, fixed-width uint64_t everywhere stored or compared.
- .dcf version 3: little-endian magic, version, longest entry in words,
uint64 count and sorted uint64 hashes. Version 2 files are refused:
the allowed words cache is rebuilt from its .txt, an old
blocklist.dcf is logged as unreadable.
- The servers build blocklist.dcf from a plain blocklist.txt next to
them (one word or phrase per line) when it is newer.
- Blocked entries can be phrases: runs of consecutive words up to the
longest entry, the whole run marked. Whitelist chat still checks one
word at a time, as the client does.
- Dashboard: the chat filter API reads blocklist.dcf the same way
(status, phrase length), accepts blocked phrases, refuses allowed
ones, and explains phrase matches in its message test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MATCH_TRANSFER is appended after GUILD_DISBAND (71); the chat ids from
CREATE_TEAM on are pinned.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Busy property instances on changed files are kept (KEEP_UNTIL_EMPTY) instead
of saved, frozen and moved. Every instance acted on is marked outdated; the
files status carries the flag (bit 4).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Property instances are left out of the replace plan. Once the database is
up to date every instance running at the start is marked outdated, and a
zone that already got a new instance is not started again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Instances on an old binary or old zone files are marked outdated: routing,
private passwords and merges skip them. Outdated properties remind their
players every 10 minutes that an update is waiting (ANNOUNCE) and stop once
empty. A new instance of the same property waits for the old one to be gone
before its world server starts, so two worlds never save one property.
The server list carries the flag after the endpoints (read when present).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Master hashes each reported file on a worker, polls size and mtime every
world_watch_seconds (a change must hold still for one poll), and replaces
stale instances with the instance migration (Mythran shift, or seamless
with world_reload_seamless=1); empty ones are stopped. Also on WORLD_RELOAD.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Appended at the end of MessageType::Master (47-49); the pin test is
updated. A world sends its zone file list to master once it is ready.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Worlds, the UGC server and the dashboard open the pair named in
resServer/cdclient-current (CDServer.sqlite without an fdb when there is
none). Worlds switch to a new pair on CDCLIENT_RELOAD between frames.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Appended at the end of MessageType::Master (46); the pin test is updated.
With names it tells a server which fdb copy and CDServer.sqlite to switch
to; without them it asks master to check the client's fdb now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FdbSnapshot names copies cdclient-<hash>.fdb / CDServer-<hash>.sqlite, writes
them under temporary names and renames them in, keeps a pointer file naming
the current pair, removes old copies, and decides when a changed file has
settled enough to hash. Nothing in it logs, so it can run on a worker.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each address's requests are counted per signed-in account (the dashboard's
session or API key, as the auth middleware found it), so several people
behind one address stay apart, with the account's user name; a WebSocket
upgrade counts under its account.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An optional section after marker 3 carries the report's frame timing; older readers stop before it and reports without it still read. Phase times go with their count so readers with fewer or more phases read them. PROFILE_REQUEST and PROFILE_RESULT are appended to the master messages (44, 45). Task 96.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Profiler.h: frames, named scopes and phases, recorded on the main thread only (other threads' scopes do nothing). Per second: frames, total and longest frame time, a mergeable frame time histogram (TrafficStats::Histogram) and time per phase. Per report: the packet types that took longest, the worst frames and the frames over a threshold with their heaviest scopes. Profiling sessions merge every frame's scope tree for a while into one (folded stacks for flame graphs). Optional Tracy client (DLU_TRACY, off by default) gets the same frames and scopes. Task 96.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FdbMappedFile maps a file read-only (CreateFileMapping/MapViewOfFile on
Windows, mmap elsewhere) and falls back to reading it into memory when
mapping fails. FdbReader reads the table and column headers from it and
looks rows up by their first column through the fdb's own buckets,
decoding every integer as little-endian with bounds checks, so the rows
never get copied out of the file.
Tests write small fdb files (collisions, text, int64, nulls) and read
them both mapped and from memory.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The login request's memoryStats is a run of short parts with no separators
(process working set and private bytes, memory load, physical memory, commit
limit, the 32-bit client's own address space, peaks). ParseMemoryStats splits
it into numbers; a text cut short keeps the parts before the cut.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every server now splits its packet counts by peer: its own connections
(players on auth and worlds), its master link, or other servers (the
worlds on chat, every server on master, a world's chat link, which is
now counted too). HTTP requests carrying X-Darkflame-Server count as
another server's, the dashboard counts its own requests to the UGC
server, and each HTTP client address is counted. The report also gets
each RakNet connection's statistics (worlds name the player on it),
trimmed to the 32 busiest with the rest summed. Counting stays on the
main loop, except the dashboard's UGC fetches, which only touch the
locked recorder.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The zone file tests' sample zones now use generic world IDs, file names
and path names instead of ones taken from real files. What they test is
unchanged.
Check in game: nothing (test only).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LDF_FROM_STRING (1.10.64, 0x010fa220) splits an object's config at
every comma and line break, leaves out empty entries and entries with
no '=', and sets each key=value it finds. The reader split at line
breaks only and parsed every piece, so an object with no config got an
empty entry of unknown type, and a value with a comma would have stayed
whole.
Live scene files are unchanged; objects in older client files with an
empty config no longer get the empty entry.
Check in game: every world spawns its objects as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ClientZoneFilesTests now
- reads sd0 files instead of skipping them: some clients' loose sd0
files hold what followed the file in its pack, and the file is the
first sd0 stream, up to the first chunk that does not fit;
- reads the terrain (.raw) of every zone of version 30 or newer, once
per terrain file shipped by several clients;
- skips a damaged file (by the MD5 of its bytes) only when it does not
read, so only the files the client cannot read either are left out:
a scene file that claims 81 objects and ends after the first, and six
version 30 terrain files with a width x width scene map per chunk
where the client reads one byte (RAWReadSceneMap, 1.10.64) and so
misreads every chunk after the first.
Every other zone, scene and terrain file of every client on disk reads,
in about 80 seconds.
Check in game: nothing (test only).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ReadLvlObjectData (1.10.64, 0x0103ba20) changes an object's config as it
loads it:
- before version 47 a respawn time is in milliseconds: a float over 100
or any u32 becomes that many seconds, as a float;
- a model spawner (LOT 176 with spawntemplate 14) gets its subkey as its
blueprintid unless it has one, DisableModelBehaviors true unless set,
and preventRenderWrapping true;
- before version 44 an object with sceneIDOverrideEnabled gets
sceneLayerIDOverride 0.
LevelFile now makes the same changes. On live worlds 1906 spawners in
114 scene files of versions 45-46 have a u32 respawn time; the world
already divided those by 1000 but as an integer, so they now keep their
fraction (a 2500 ms respawn is 2.5 s, not 2 s). No live scene has a
model spawner; 62 objects of version 43 scenes get sceneLayerIDOverride.
Check in game: enemies and smashables respawn after the same time as
before on Forbidden Valley, Avant Gardens and Gnarled Forest.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SceneLoader::ReadLvlFile (1.10.64):
- chunked files: the file info chunk starts the file and gives where
the environment, object and particle chunks start, 0 for none
(DoLvlChunk, ReadLvlChunk1000); the reader walked the chunks one after
another by their sizes instead;
- older files: the editor settings are a u32 size and that many bytes
(SceneLoader::ReadEditorSettings, skipped by size); the reader took
them for a u32 and then a count of 12 byte points;
- older files before version 3 have no objects for the client ("Level
file is unsupported").
LevelFile now does the same. Every scene file on disk reads the same
objects as before.
Check in game: every world spawns its objects as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ReadLvlObjectData (1.10.64, 0x0103ba20) reads a scene object's last
field, from version 7 on, as a render technique count, and when it is
not 0 a 64 byte block and 133 bytes per technique (a 64 byte name, a
u32, a u8 and 16 floats) follow. The reader took the count for an
unknown u32 and read the next object from inside the techniques, and
read the u32 for every version. It also left the node type and glom ID
uninitialized when a file has none (or, for the node type, one outside
0-10); the client uses 1 for both.
The count is now SceneObject::renderTechniqueCount, the techniques are
skipped, and the defaults are the client's. Every scene file on disk
has no render techniques, so what they read is unchanged.
Check in game: every world spawns its objects as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LevelPath::FromBuffer (1.10.64) changes two values as it reads a path:
- a property path's reputation multiplier outside 0-1000 becomes 0;
- a camera waypoint's FOV in a path older than version 11 is divided
by 1.25.
The reader kept both as written. Only camera paths of older client
files (versions 8-10) change; no live zone has either case.
Check in game: nothing to check (no live zone changes).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The client reads a spawner path's fields (LOT, respawn time, counts,
object ID) and a camera path's next path and rotate-player flag only
when the path is newer than version 3 (LevelPath::FromBuffer, 1.10.64:
pathVersion > 3). The reader read them for every version, which would
misread an older spawner or camera path from there on.
No zone file on disk has a spawner or camera path older than version 4,
so what they read is unchanged.
Check in game: nothing to check (no live zone changes).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The client reads a zone file's paths as a chunk of its own
(LuzFile::ReadLUZFile copies the u32-sized chunk, ReadLUZPaths reads
it, 1.10.64) and drops every path when it refuses any of it:
- a chunk version of 2 or more, or 10000 paths or more (ReadLUZPaths);
- a path of version 19 or more, 10000 waypoints or more, or a
waypoint with more than 99 name/value pairs (LevelPath::FromBuffer).
The reader read the paths straight from the file, trusting the chunk to
be well formed. It now reads the chunk by its size, reads the paths
from it with the client's limits, and when one is refused the zone has
no paths (logged), while the rest of the zone file still reads.
Every zone file on disk reads the same paths as before.
The unit tests' sample zones now give the path chunk its real length.
Check in game: moving platforms, NPC patrols and spawners work as
before on every world.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Scene transitions of versions 34-38 have five points. The client links
the scenes of the first and the last one (ZoneLoader::ReadZoneFile,
1.10.64: points[0] and points[count - 1], for every version); the scene
graph used the first two, so in those zones it linked a scene with the
one of the transition's second point instead.
Of the 130 five-point transitions on disk, six (all in one older client
zone) have a different scene at the second and last points; the rest
link the same scenes as before.
Live zones are unchanged.
Check in game: nothing to check (no live zone changes).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LuzReader::ReadScenes (1.10.64) keeps a zone's scenes by scene ID and
layer, so a later scene with the same ID and layer replaces an earlier
one. ZoneLoader::ReadZoneFile then goes through them in scene ID order
and
- leaves out every layer of a scene ID that has no layer 0 (General)
scene;
- before version 41 loads only the layer 0 scene of each ID; from 41 on
the other layers (Audio, FX) are loaded with it.
ZoneFile now keeps the scenes the same way, so the world loads the same
scenes as the client. No zone file on disk has a duplicate, a scene
without a General layer, or a non-General layer before version 41, and
their scenes are already in ID order: what every file reads is
unchanged.
Check in game: nothing to check (no live zone changes).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Zone files before version 33 have no scene IDs. The client gives each
scene its index in the file as its ID, -1 past 255, with layer 0
(LuzReader::ReadScenes, 1.10.64, via the LWOSceneID from the loop
index); the reader left every such scene at ID 0, so a zone with several
scenes had them all collide on one ID.
The version 30 zones on disk have one scene each, so what they read is
unchanged.
Check in game: nothing to check on live worlds (all are version 36+).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Zone files older than version 30 were refused. The client reads them
(1.10.64):
- LuzFile::ReadLUZFile reads a version below 20 as 20;
- LuzReader::ReadScenes reads a scene of a version 20-29 file as a bare
u32 SceneTable ID, with no file name;
- ZoneLoader::ReadZoneFile takes those scenes in SceneTable ID order,
looks each one up in the CDClient SceneTable and, when there is a
row, makes it a scene with the next index (0, 1, ...; -1 past 255) as
its ID and the row's sceneName as its file; a scene with no row is
left out.
ZoneFile now reads these files (FileFormatVersion::Oldest = 20 for
anything older), keeps the SceneTable ID on ZoneScene::sceneTableID,
and ZoneFile::ResolveSceneTable does the lookup the client does; the
world looks the names up in the CDClient SceneTable.
The only such file on disk (version 20) now reads.
Check in game: nothing to check on live worlds (all are version 36+).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LWOSCENEID and LWOSCENEID_INVALID move from dCommonVars.h to
dZoneManager/LWOSCENEID.h; only the zone code uses them. The layer half
is the scene's layer, now the new eSceneType (dCommon/dEnums) instead of
a uint32_t: General (0), Audio (1, the *_audio.lvl scenes named
"Audio") and FX (2, Nexus Tower's *_fxs.lvl scenes named "FXs"), the
three layers the zone files of every client on disk use. ZoneScene's
sceneType is an eSceneType too, so a scene's LWOSCENEID is built from it
directly, and the dashboard compares against eSceneType::General instead
of 0. The zone checksum still hashes the layer's number.
Check in game: every world loads (the zone checksum the client checks is
unchanged), Nexus Tower included.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ClientZoneFilesTests reads every .luz and .lvl under DLU_CLIENTS_DIR
(default ~/Documents/luclients) and expects each to read in full; the
tests are skipped when no client is there. Left out: empty files, sd0
files (the loose sd0 files of the 1.7.45 and 1.9.76 clients are all cut
up after their second chunk; the server gets its files uncompressed from
the packs) and one damaged scene file, which claims 81 objects
in 910 bytes.
With the zone and scene reader fixes before this every file of the
0.179.12, 1.7.45, 1.9.76, 1.10.64 and unpacked clients
reads.
Check in game: nothing (test only).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two fields of the scene files from before chunks were read for versions
the client does not read them for (SceneLoader::ReadLvlFile, 1.10.64):
- the "important" byte after the version and type is there only from
version 32 (SceneLoader::ReadLvlHeader, 0x010117d0: headerVersion < 32 sets it to 0);
- the five strings after the skydome's file are there only from version
34 (SceneLoader::ReadSkydomeInfo, 0x0102f550: headerVersion > 33); the reader took
them from version 33.
Seven scenes of the alpha leak (versions 31 and 33: Gnarled_assetsMIKET,
dennis_gnarled_forest_02/_03-ninja/_03-pirate_0, scale_TEST_2,
scale_test_2_0, scale_test_general) now read with all their objects.
Every other scene file on disk reads the same objects as before.
Check in game: every world spawns its objects as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The client reads a spawner path's activateOnLoad byte only when the
path is newer than version 8 (LevelPath::FromBuffer, 1.10.64:
pathVersion > 8) and otherwise keeps its default of 1
(Path::InitializeSpawnerInfo). The reader read the byte for every
version, which would misread an older spawner path from there on, and
its default was 0. The byte is now read from version 9, and an older
path's spawner is active on load as in the client.
No zone file on disk has a spawner path older than version 9, so what
the world spawns is unchanged.
Check in game: nothing to check (spawners on live worlds are unchanged).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Paths before version 3 are laid out differently (LevelPath::FromBuffer,
1.10.64: pathVersion < 3):
- the type is a u8-length wide string, "platform" for a moving platform
and anything else ("npc") for a movement path, instead of a u32;
- the flags and behavior u32s follow as in later versions;
- every waypoint, whatever the path's type, has a rotation, lock player
byte, speed and wait time and then its name/value pairs.
The reader treated the type name's first bytes as the type and misread
everything after it. The LUP group zones of the 0.179.12
client (lup_group_1b, _2a, _6a) now read with all their paths; one of
them read before with a garbage type on a waypoint-less path. Every
other zone reads the same as before (no live zone has a path older than
version 3).
Check in game: nothing to check on live worlds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
