Master copies the fdb at startup, polls its size and mtime every
cdclient_watch_seconds, builds the new copy and CDServer.sqlite on a
worker, switches itself, writes the pointer file, tells every world and
keeps the last two copies.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Worlds, the UGC server and the dashboard open the pair named in
resServer/cdclient-current (CDServer.sqlite without an fdb when there is
none). Worlds switch to a new pair on CDCLIENT_RELOAD between frames.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Appended at the end of MessageType::Master (46); the pin test is updated.
With names it tells a server which fdb copy and CDServer.sqlite to switch
to; without them it asks master to check the client's fdb now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CDClientSnapshot converts a copy of the client's fdb into its own
CDServer-<hash>.sqlite with the cdserver migrations applied, on its own
connection. CDClientDatabase::Reconnect opens the new file before letting go
of the old one; CDClientManager::Reload empties every table (old entries kept
alive so references from spawned objects stay valid), retires the old fdb view
and loads again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
So a worker can make a new CDServer.sqlite without touching the shared
CDClient connection or the logger.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FdbSnapshot names copies cdclient-<hash>.fdb / CDServer-<hash>.sqlite, writes
them under temporary names and renames them in, keeps a pointer file naming
the current pair, removes old copies, and decides when a changed file has
settled enough to hash. Nothing in it logs, so it can run on a worker.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Launching to a property started an extra clone 0 instance of the zone besides the property's own, because
the prep only carried the zone. PrepZone now carries the clone when there is one (written only then, so a
plain prep is unchanged).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fixture check now also reads every recorded client game message with the
struct the server reads it with (GameMessageHandler::CreateReceived) and writes
it again: it must read the whole message and give back the same bits. Game
message fields are decoded with the server's structs in the fixture tests.
A synthetic fixture, built in the test from the server's own structs (login,
position update, a client game message), goes through the export steps
(portable, anonymised, saved, read again) and passes the same checks; recorded
fixtures stay in tests/fixtures-local and are never committed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GameDependenciesTest deleted the logger, config and managers but left the
pointers set, so a later test without the fixture (packet capture) logged
through a freed logger and crashed when the whole suite ran.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sandbox launched its servers with fork, setpgid and waitpid only. Starting,
waiting and stopping now go through one small platform section: POSIX keeps
the process group, Windows uses CreateProcess in a job object that is ended as
one (and dies with the tool). Environment overrides are cleared in the tool
itself, program names get .exe on Windows, and shared folders fall back to a
copy where links can't be made.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- the expected answers pointed into a copy of the records that was gone by the diff
- the client's recorded connect and disconnect messages are RakNet's, not sent again
- a recorded move to another world is waited for as long as a zone takes to start
- the handshake sends the client net version; reports are written after every bundle
- a crash or Ctrl-C of the tool stops the sandbox stack too
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CaptureTool (built next to the servers) replays packet bundles and compares the answers:
- replay: per bundle a fresh sandbox folder with copied server binaries, rewritten settings
(replay_sandbox=1, a new SQLite file inside the folder, ports from --port on, no dashboard),
read back before anything starts; sandbox-setup runs inside it to apply migrations and make
the replay account and the bundle's characters (setup mode); master is started, the stack is
stopped as one process group and the folder deleted unless kept
- with replay_sandbox=1 every server refuses a database that isn't SQLite, isn't inside its
own folder, or is replay_live_sqlite_path (Database::Connect); replay-target against a
running server needs --i-know-this-is-not-a-sandbox
- the fake client splits the recording into connections, logs in and picks the character
itself when the recording doesn't, fills in the target's account, session key and IDs,
learns server-made object IDs from replica constructions by LOT, follows the recorded
timing and waits for the answers a client waits for; the diff pairs answers by name (and
constructions by LOT) and ignores fields that differ between runs
- import-live converts the 2014 live captures (folders of *_traffic.zip; pcaps and encrypted
captures are left alone) into bundles, with secrets removed and CREATE_CHARACTER as setup
- anonymise makes local fixtures; docs/CaptureReplay.md describes capture, the bundle format,
portability rules, the sandbox and the replay
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Packet Captures page (/inspector/packets, dev_message_inspector) arms captures with a
search picker (part of an account or character name, or a pasted account or object ID; online
ones marked; everything needs no pick), lists running and saved ones live (packet_capture
topic), and plays one back on its timeline: play, pause, seek and speed, packets appearing in
order with their decoded fields and bytes. World 3D takes ?capture=<id>&zone=<zone> and replays
the captured position updates in its replay mode; the capture page's playhead drives it over a
BroadcastChannel. Saved packet captures show in the Message Inspector's list and open there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Staff arm a packet capture on the dashboard; master passes MESSAGE_CAPTURE_CONTROL ARM to
every world, auth and chat and arms its own. Each server's PacketCapture tap (dServer receive,
and a send hook in RakPeer::Send so replica constructions are seen too) records into one
preallocated chunk per server and ships sealed chunks through master on the main loop when
capture_flush_bytes or capture_flush_interval_ms is reached; past capture_buffer_max_mb the
oldest chunks are dropped and the dashboard records a gap. Nothing is armed: one flag check.
- targets: an account (from its login; packets before the login are kept per connection
and added once auth or the world knows whose they are), a character (from when it is
picked), or everything; up to 8 at once (a bit each in the record mask)
- worlds and auth record their clients' packets and the master link messages of a captured
player (session keys by name, zone transfers by request, player added/removed, migration);
chat finds the player in each packet; master records server traffic for everything
- secrets are never recorded: structs that carry them (login request, login response user
key, world validation session key, session key messages between servers) are read,
blanked and written again before recording; auth keeps only the handshake and login
- PacketDecoder: a registry by service and message id names every packet and decodes the
registered structs; CaptureBundle is the file format (DLUBNDL1, metadata, records);
CaptureTools orders records on one timeline, pulls movement out, makes bundles portable
or anonymous and diffs replays
- the dashboard keeps packet captures in message_capture_sessions (capture_kind 1) and
their packets in a file under capture_dir, one write per batch; arming is audited
- MESSAGE_CAPTURE_CONTROL/DATA only gain appended enum values and trailing fields
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An open group (game clients, web clients, a zone's instances) keeps one box
that grows by its rows, at most 8 in view with the rest scrolled. Each row
in view is a link end of its own with only its member's traffic, and the
open box's header has none. Members are filtered by name, account or user,
instance and (with network_ips) address, in a steady order; web clients
show the dashboard user.
The filter and rows are HTML over the SVG, made once and then moved and
updated in place, so live updates keep the typed text, the focus and the
scroll; links follow the list's scroll. The box grows and shrinks with its
list, and the List view nests the members under their group with the same
filter.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Web clients are one entry per address and signed-in account, with user and
account_id; the user is shown without network_ips too, the address then
masked as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each address's requests are counted per signed-in account (the dashboard's
session or API key, as the auth middleware found it), so several people
behind one address stay apart, with the account's user name; a WebSocket
upgrade counts under its account.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
How frame timing is measured and reported, the page, the API and connecting Tracy's viewer. Task 96.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dashboard's own loop is framed too, with a scope per module update. Frame time and stacked phase charts per server, the servers' loop summary, longest frames, packet handling times, the last 50 slow frames with a nested timeline, and profiling sessions (profiling_run, GM 8) drawn as a flame graph with folded stacks to download. PerfHistory keeps it in memory and is unit tested; the layouts are tested with node. Task 96.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Frames and phase scopes in the world, auth, chat and UGC loops (packets per type, entities, physics, ghosting, replica, spawners, log flush, saves, web requests by route). Scopes at LoadPlayer, CreateEntity, each component's construction, InventoryComponent::LoadXml, script timers and a world's zone load; game database queries in the query helpers; CDClient statements timed through sqlite3_trace_v2 as CDClient <table> (CppSQLite3DB gets a handle accessor). Task 96.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dServer sends the frames with each traffic report, logs a frame over slow_frame_ms (default 250, reread on a settings reload) as one line with its heaviest path, and runs profiling sessions master forwards. Master routes the dashboard's requests to the named server, profiles itself and passes results on. Task 96.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An optional section after marker 3 carries the report's frame timing; older readers stop before it and reports without it still read. Phase times go with their count so readers with fewer or more phases read them. PROFILE_REQUEST and PROFILE_RESULT are appended to the master messages (44, 45). Task 96.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Profiler.h: frames, named scopes and phases, recorded on the main thread only (other threads' scopes do nothing). Per second: frames, total and longest frame time, a mergeable frame time histogram (TrafficStats::Histogram) and time per phase. Per report: the packet types that took longest, the worst frames and the frames over a threshold with their heaviest scopes. Profiling sessions merge every frame's scope tree for a while into one (folded stacks for flame graphs). Optional Tracy client (DLU_TRACY, off by default) gets the same frames and scopes. Task 96.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GameTextJs scans templates, scripts and the routes' strings for the
game's zone, item and character names and currency labels, with an
allowlist for legitimate uses and a self check of the scanner.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pages label the game's currencies and stats with the client's words
(UI_COINS, UI_UNIVERSE_SCORE, UI_IMAGINATION, UI_REPUTATION, UI_HEALTH)
through game.terms and GameText.term instead of writing them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Settings help names zones by locale key (expanded when served), the
Property Rent page and the Worlds table name zones in the viewer's
language, and the UGC page drops its own inventory name table.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The routes' own zone name helpers and the CDClient's English zone and
leaderboard header copies give way to GameText, so every name comes from
the locale in the viewer's language; the Network page's world labels,
mail, missions, leaderboards, item info (cached per language) and the
settings/vanity help strings too. Reward code 4 and the plaque text name
their zone from the locale.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dashboard loads every language in the client's locale; each request
runs in the viewer's language (Workers::Reply carries it to the worker),
templates get game.* and phrase()/zone_name(), scripts get GameText.* on
<body data-game>, and the user menu picks the game text language.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One helper for game text on the dashboard: zone names, locale phrases,
localized table columns, %[key] expansion and the game's currency words,
looked up in the viewer's language (a cookie pick, else Accept-Language),
falling back to en_US and then the key or id. Unit tested.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Locale::LoadFromFile can keep the other locales' phrases too, and
GetPhrase(id, locale) answers in one of them, falling back to the default.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Skipped unless DLU_FDB_BENCH_XML names a character save; prints startup
time, inventory load time and resident memory (private and file-backed,
on Linux) for DLU_FDB_BENCH_MODE=fdb or sqlite.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The world and master servers pass the client's res/cdclient.fdb to
CDClientManager. When it opens, these three tables, all looked up by
their first column, find rows through the fdb's buckets instead of each
process caching the whole table: ComponentsRegistry keeps nothing,
ItemComponent and Objects keep only the entries asked for (their API
returns references). Ids whose rows CDServer.sqlite changes are loaded
from SQLite at startup and win. Without an fdb (or with one whose
columns don't match) the tables load from CDServer.sqlite as before.
ItemComponent and Objects now fill entries from one template for both
sources instead of copies of the same field list.
Tests cover the SQLite changes on top of the fdb, the no-fdb and
unmapped paths, and, when DLU_CLIENT_RES points at a client, every id of
the three tables through the fdb against CDServer.sqlite.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CDFdb opens cdclient.fdb once per process and hands out a table when its
columns match the CDServer.sqlite table of the same name. CDServer.sqlite
stays the source of truth: the CDServer migrations change a few rows, so
FindChangedKeys compares both files row by row (a hash per row, summed
per key) and returns the keys that differ, for the tables to read from
SQLite. RowFields reads an fdb row with CppSQLite3Query's accessors and
defaults, so a table fills its entries from either file with one piece
of code.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FdbMappedFile maps a file read-only (CreateFileMapping/MapViewOfFile on
Windows, mmap elsewhere) and falls back to reading it into memory when
mapping fails. FdbReader reads the table and column headers from it and
looks rows up by their first column through the fdb's own buckets,
decoding every integer as little-endian with bounds checks, so the rows
never get copied out of the file.
Tests write small fdb files (collisions, text, int64, nulls) and read
them both mapped and from memory.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The client's TacArcBehavior::Cast (0x00fb2d10) sorts the targets in the arc nearest first, or by weight when
distance_weight or angle_weight is set (sortWithWeights, 0x00f58cd0: distance_weight * (max range - distance) /
max range + angle_weight * (180 - angle) / 180, heaviest first). With use_attack_priority, SortByAttackPriority
(0x00f72900) then buckets them by GetAttackPriority, lowest first, keeping that order inside each bucket; only the
DestroyableComponent answers it, and an object without one counts as 1. DoHit keeps the first max targets.
Nothing else ranks targets: enemies are taken over nearer smashables only because their attack_priority (1) is
lower than most smashables' (10). use_attack_priority is off when a behavior does not set it (TacArcBehavior::
Initialize, 0x00f9b980), as before.
The server sorted by distance only and ignored the flag, so a one-target swing hit the nearest crate instead of the
enemy behind it. OrderTargets now does the client's ordering, with equal targets in ascending id order (the order
the client's id set hands them over in); the chosen targets are still written in ascending id order (issue 1045).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DestructibleComponent.attack_priority is loaded onto the DestroyableComponent as a signed value. The client's
LWODestroyableComponent (LoadDataFromTemplate, 0x00c9f900) starts it at 1 and keeps 1 when the column is empty, and
answers GetAttackPriority with it; the server now does the same so TacArcs can order their targets by it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every entity and every inventory item looks these up, and an id not seen yet was a full scan of the
unindexed table, so a character holding about 3000 different items took a minute to load (the client
waited at 35%). Loading both tables once at startup costs a few MB per server.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The converted CDClient had no indexes, so each lazy lookup scanned its table (ComponentsRegistry: about
51000 rows, 3.5 ms a query; 0.09 ms with the index). Loading a player with a large inventory spent up to
a minute of the world's main loop in these scans, leaving the client at 35%.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
