Commit Graph

2559 Commits

Author SHA1 Message Date
Aaron Kimbrell
fb6d73e4bd feat(web): API key traffic is its own Network row, named after the key
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
355d997333 docs(dashboard): Network groups open in place; web clients by signed-in user
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
c57ef49b15 feat(dashboard): Network groups open in place as a filterable list in their box
An open group (game clients, web clients, a zone's instances) keeps one box
that grows by its rows, at most 8 in view with the rest scrolled. Each row
in view is a link end of its own with only its member's traffic, and the
open box's header has none. Members are filtered by name, account or user,
instance and (with network_ips) address, in a steady order; web clients
show the dashboard user.

The filter and rows are HTML over the SVG, made once and then moved and
updated in place, so live updates keep the typed text, the focus and the
scroll; links follow the list's scroll. The box grows and shrinks with its
list, and the List view nests the members under their group with the same
filter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
2fb68cd20e feat(dashboard): Network connections name the web client's dashboard user
Web clients are one entry per address and signed-in account, with user and
account_id; the user is shown without network_ips too, the address then
masked as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
569ac6fb31 feat(web): count HTTP clients by the account they were signed in as
Each address's requests are counted per signed-in account (the dashboard's
session or API key, as the auth middleware found it), so several people
behind one address stay apart, with the account's user name; a WebSocket
upgrade counts under its account.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
7c37fca9fb docs(dashboard): Performance page, slow frames, profiling and Tracy
How frame timing is measured and reported, the page, the API and connecting Tracy's viewer. Task 96.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
992abf9d8a feat(dashboard): Performance page with frame times, slow frames and flame graphs
The dashboard's own loop is framed too, with a scope per module update. Frame time and stacked phase charts per server, the servers' loop summary, longest frames, packet handling times, the last 50 slow frames with a nested timeline, and profiling sessions (profiling_run, GM 8) drawn as a flame graph with folded stacks to download. PerfHistory keeps it in memory and is unit tested; the layouts are tested with node. Task 96.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
d4e9423264 feat(servers): time the main loops' phases and the known heavy spots
Frames and phase scopes in the world, auth, chat and UGC loops (packets per type, entities, physics, ghosting, replica, spawners, log flush, saves, web requests by route). Scopes at LoadPlayer, CreateEntity, each component's construction, InventoryComponent::LoadXml, script timers and a world's zone load; game database queries in the query helpers; CDClient statements timed through sqlite3_trace_v2 as CDClient <table> (CppSQLite3DB gets a handle accessor). Task 96.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
63666c3c6b feat(servers): report frame timing, log slow frames, answer profiling requests
dServer sends the frames with each traffic report, logs a frame over slow_frame_ms (default 250, reread on a settings reload) as one line with its heaviest path, and runs profiling sessions master forwards. Master routes the dashboard's requests to the named server, profiles itself and passes results on. Task 96.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
d0c7b089ff feat(net): frame timing section in SERVER_TRAFFIC, profiling messages
An optional section after marker 3 carries the report's frame timing; older readers stop before it and reports without it still read. Phase times go with their count so readers with fewer or more phases read them. PROFILE_REQUEST and PROFILE_RESULT are appended to the master messages (44, 45). Task 96.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
06af6c3aaa feat(profiler): frame timing and scope trees of a server's main loop
Profiler.h: frames, named scopes and phases, recorded on the main thread only (other threads' scopes do nothing). Per second: frames, total and longest frame time, a mergeable frame time histogram (TrafficStats::Histogram) and time per phase. Per report: the packet types that took longest, the worst frames and the frames over a threshold with their heaviest scopes. Profiling sessions merge every frame's scope tree for a while into one (folded stacks for flame graphs). Optional Tracy client (DLU_TRACY, off by default) gets the same frames and scopes. Task 96.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:26:17 -05:00
Aaron Kimbrell
6758b32605 docs(dashboard): game text and languages
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:47 -05:00
Aaron Kimbrell
6230d35518 test(dashboard): fail when game text is written into the pages
GameTextJs scans templates, scripts and the routes' strings for the
game's zone, item and character names and currency labels, with an
allowlist for legitimate uses and a self check of the scanner.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:47 -05:00
Aaron Kimbrell
597269b4c5 fix(dashboard): coins, universe score, imagination, reputation and life from the locale
Pages label the game's currencies and stats with the client's words
(UI_COINS, UI_UNIVERSE_SCORE, UI_IMAGINATION, UI_REPUTATION, UI_HEALTH)
through game.terms and GameText.term instead of writing them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:47 -05:00
Aaron Kimbrell
ade116c985 fix(dashboard): zone names in help text and pages come from the locale
Settings help names zones by locale key (expanded when served), the
Property Rent page and the Worlds table name zones in the viewer's
language, and the UGC page drops its own inventory name table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:47 -05:00
Aaron Kimbrell
1f8dd73e0b refactor(dashboard): routes name zones, objects and phrases through GameText
The routes' own zone name helpers and the CDClient's English zone and
leaderboard header copies give way to GameText, so every name comes from
the locale in the viewer's language; the Network page's world labels,
mail, missions, leaderboards, item info (cached per language) and the
settings/vanity help strings too. Reward code 4 and the plaque text name
their zone from the locale.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:47 -05:00
Aaron Kimbrell
17b9479c0e feat(dashboard): every request and page gets game text in the viewer's language
The dashboard loads every language in the client's locale; each request
runs in the viewer's language (Workers::Reply carries it to the worker),
templates get game.* and phrase()/zone_name(), scripts get GameText.* on
<body data-game>, and the user menu picks the game text language.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:46 -05:00
Aaron Kimbrell
a6fc67a4bb feat(dashboard): GameText, the game's text in the viewer's language
One helper for game text on the dashboard: zone names, locale phrases,
localized table columns, %[key] expansion and the game's currency words,
looked up in the viewer's language (a cookie pick, else Accept-Language),
falling back to en_US and then the key or id. Unit tested.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:46 -05:00
Aaron Kimbrell
40d9364770 feat(locale): load every locale in locale.xml on request
Locale::LoadFromFile can keep the other locales' phrases too, and
GetPhrase(id, locale) answers in one of them, falling back to the default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:46 -05:00
Aaron Kimbrell
8cbae8bb22 test(cdclient): benchmark startup and a big inventory with and without the fdb
Skipped unless DLU_FDB_BENCH_XML names a character save; prints startup
time, inventory load time and resident memory (private and file-backed,
on Linux) for DLU_FDB_BENCH_MODE=fdb or sqlite.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:46 -05:00
Aaron Kimbrell
a4fa7eb30a perf(cdclient): read ComponentsRegistry, ItemComponent and Objects from the fdb
The world and master servers pass the client's res/cdclient.fdb to
CDClientManager. When it opens, these three tables, all looked up by
their first column, find rows through the fdb's buckets instead of each
process caching the whole table: ComponentsRegistry keeps nothing,
ItemComponent and Objects keep only the entries asked for (their API
returns references). Ids whose rows CDServer.sqlite changes are loaded
from SQLite at startup and win. Without an fdb (or with one whose
columns don't match) the tables load from CDServer.sqlite as before.

ItemComponent and Objects now fill entries from one template for both
sources instead of copies of the same field list.

Tests cover the SQLite changes on top of the fdb, the no-fdb and
unmapped paths, and, when DLU_CLIENT_RES points at a client, every id of
the three tables through the fdb against CDServer.sqlite.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:46 -05:00
Aaron Kimbrell
17eb26fb1d feat(cdclient): open the client's fdb next to CDServer.sqlite
CDFdb opens cdclient.fdb once per process and hands out a table when its
columns match the CDServer.sqlite table of the same name. CDServer.sqlite
stays the source of truth: the CDServer migrations change a few rows, so
FindChangedKeys compares both files row by row (a hash per row, summed
per key) and returns the keys that differ, for the tables to read from
SQLite. RowFields reads an fdb row with CppSQLite3Query's accessors and
defaults, so a table fills its entries from either file with one piece
of code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:45 -05:00
Aaron Kimbrell
d670484b68 feat(dCommon): read cdclient.fdb in place through its hash buckets
FdbMappedFile maps a file read-only (CreateFileMapping/MapViewOfFile on
Windows, mmap elsewhere) and falls back to reading it into memory when
mapping fails. FdbReader reads the table and column headers from it and
looks rows up by their first column through the fdb's own buckets,
decoding every integer as little-endian with bounds checks, so the rows
never get copied out of the file.

Tests write small fdb files (collisions, text, int64, nulls) and read
them both mapped and from memory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:07:45 -05:00
Aaron Kimbrell
94b411a32e feat(combat): TacArc picks its targets in the client's order
The client's TacArcBehavior::Cast (0x00fb2d10) sorts the targets in the arc nearest first, or by weight when
distance_weight or angle_weight is set (sortWithWeights, 0x00f58cd0: distance_weight * (max range - distance) /
max range + angle_weight * (180 - angle) / 180, heaviest first). With use_attack_priority, SortByAttackPriority
(0x00f72900) then buckets them by GetAttackPriority, lowest first, keeping that order inside each bucket; only the
DestroyableComponent answers it, and an object without one counts as 1. DoHit keeps the first max targets.

Nothing else ranks targets: enemies are taken over nearer smashables only because their attack_priority (1) is
lower than most smashables' (10). use_attack_priority is off when a behavior does not set it (TacArcBehavior::
Initialize, 0x00f9b980), as before.

The server sorted by distance only and ignored the flag, so a one-target swing hit the nearest crate instead of the
enemy behind it. OrderTargets now does the client's ordering, with equal targets in ascending id order (the order
the client's id set hands them over in); the chosen targets are still written in ascending id order (issue 1045).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 21:44:48 -05:00
Aaron Kimbrell
e017c11aa2 feat(combat): destroyables keep their attack priority
DestructibleComponent.attack_priority is loaded onto the DestroyableComponent as a signed value. The client's
LWODestroyableComponent (LoadDataFromTemplate, 0x00c9f900) starts it at 1 and keeps 1 when the column is empty, and
answers GetAttackPriority with it; the server now does the same so TacArcs can order their targets by it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 21:44:48 -05:00
Aaron Kimbrell
044933ebee docs(readme): the branch's guilds, chat history and flags, Network page, UGC processing options, loot, enemy walls and build identifier
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 20:55:06 -05:00
Aaron Kimbrell
3a5c74fcd1 perf(cdclient): keep ComponentsRegistry and ItemComponent in memory
Every entity and every inventory item looks these up, and an id not seen yet was a full scan of the
unindexed table, so a character holding about 3000 different items took a minute to load (the client
waited at 35%). Loading both tables once at startup costs a few MB per server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:44:44 -05:00
Aaron Kimbrell
cdd52aba5d Revert "perf(cdclient): index the columns the lazy lookups search by"
This reverts commit 3780ce042b.
2026-09-29 19:41:55 -05:00
Aaron Kimbrell
3780ce042b perf(cdclient): index the columns the lazy lookups search by
The converted CDClient had no indexes, so each lazy lookup scanned its table (ComponentsRegistry: about
51000 rows, 3.5 ms a query; 0.09 ms with the index). Loading a player with a large inventory spent up to
a minute of the world's main loop in these scans, leaving the client at 35%.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:38:46 -05:00
Aaron Kimbrell
381800fe5e feat(dashboard): Network groups open and close smoothly
A labelled button (▸ 3 instances, ▸ each, ▾ group) or a double click opens a group; its members stack
where the group is (or was dragged to), boxes glide to their new places and new ones fade in. Rows keep
a fixed pitch, so opening a group only moves its own column, and open groups are remembered per browser.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:30:31 -05:00
Aaron Kimbrell
0787e651a6 docs: client system info and what each field is worth
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:09:22 -05:00
Aaron Kimbrell
c44af311ad feat(dashboard): client system info, as reported by the client
Account pages get a Client system info (as reported) card (client_sysinfo):
every description the account's client sent at login, newest first, with the
raw values, the memory text split into numbers and each field's caveat. The
address is only shown with logs_audit. The Client System Info page (Logs &
Health) shows the spread across players from each account's newest report:
Windows version, video card, memory buckets, processor count and client build,
marked as client-reported and approximate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:09:22 -05:00
Aaron Kimbrell
05a99cef82 feat(dashboard): client_sysinfo permission and retention
client_sysinfo (GM 5, grantable) shows the client system info; the Log pruning
task deletes rows not seen for log_client_sysinfo_days (90).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:09:14 -05:00
Aaron Kimbrell
55775d9623 feat(auth): keep the client's system info at login
After a successful login the auth server stores the system description the
client sent (memory text, video card, processor fields, Windows version fields)
exactly as sent, on the main thread with the login's other writes. The address
is only kept while log_login_addresses is on; log_client_sysinfo (on) turns the
whole thing off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:09:14 -05:00
Aaron Kimbrell
7bc41f2246 feat(db): client system info as reported by the client
client_sysinfo (migrations mysql 102, sqlite 85) keeps the system description
each account's client sent at login, exactly as sent, plus the physical memory
read from it. While nothing but the memory in use changes, the account's newest
row gets the new time and one more login; otherwise a new row starts. Log
pruning deletes rows not seen for a while (eLog::CLIENT_SYSINFO) and deleting
an account deletes its rows.

Tests on SQLite alone (dDatabaseSqliteTests) and in the MySQL parity tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:09:14 -05:00
Aaron Kimbrell
b67e0ba9ba feat(common): read the memory text the client sends at login
The login request's memoryStats is a run of short parts with no separators
(process working set and private bytes, memory load, physical memory, commit
limit, the 32-bit client's own address space, peaks). ParseMemoryStats splits
it into numbers; a text cut short keeps the parts before the cut.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:09:13 -05:00
Aaron Kimbrell
1e5660512e refactor(chat): remove the chat server's web API
The dashboard's API covers what it offered (online players, teams, announcements) with API keys, so the
chat server no longer runs a web server; its settings show as unused on the settings page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:09:02 -05:00
Aaron Kimbrell
8948e2e81a fix(dashboard): Network links follow dragged boxes; Diagram or List, no Auto
Links leave and enter on the sides that face each other (top and bottom when one box is above the
other), so moving a box no longer leaves links ending in the air.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:07:27 -05:00
Aaron Kimbrell
9b00e437e8 fix(dashboard): Network connections are one entry per player or per connection, not per address
A player on the servers' machine (or behind the same address as other players) was merged with the
servers' own links and everyone else there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:07:27 -05:00
Aaron Kimbrell
a824ff9914 feat(dashboard): drag the Network page's boxes around; links coloured against the busiest link
The layout the viewer makes is kept in the browser, with a Reset layout button. A link's colour now
compares it with the busiest link right now instead of its own recent peak, which made steady links
look saturated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:41:35 -05:00
Aaron Kimbrell
6a59da1748 fix(dashboard): web clients sit right of the dashboard on the Network page
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:38:17 -05:00
Aaron Kimbrell
c9774173bb docs(dashboard): the Network page and traffic by peer
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:28:39 -05:00
Aaron Kimbrell
e6b7677c1f feat(dashboard): Network page with a live traffic flow diagram
Next to Diagnostics (health_view): game and web clients, auth, chat,
worlds per zone (expandable), other reporting servers, master, the
dashboard and the UGC server, with a lane each way per link sized by
bytes/s, animated by packets/s and coloured by load against its own
5 minute peak. Estimated links (servers without a split) are dashed.
Clicking a box shows its links, busiest message types and a sparkline,
with a link to Diagnostics (which now takes ?server=). A connections
table lists each remote address; single clients can be drawn too.
Narrow screens get a list. Drawing stops while the tab is hidden.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:28:39 -05:00
Aaron Kimbrell
7e87f2a482 feat(dashboard): network summary, server detail and connection routes
The traffic topic now carries each server's rates with its split by
peer (null for servers that report none), link statistics and gauges.
New routes: /api/diagnostics/network, /network/server (message types
and a 10 minute series) and /network/connections (remote ends grouped
by address). Addresses need the new network_ips permission; without it
each is a salted token. They stay in memory from the last report only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:28:39 -05:00
Aaron Kimbrell
3b778065cf feat(traffic): send the peer split and connections in SERVER_TRAFFIC
Two optional sections at the end of the report, each after a marker
byte: every second's packets by peer with its HTTP requests from and to
other servers, and the busiest remote ends with the rest summed.
Reports without them still read (older servers), and older readers stop
before them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:28:39 -05:00
Aaron Kimbrell
9e8f161dd8 feat(traffic): count traffic by peer and by connection
Every server now splits its packet counts by peer: its own connections
(players on auth and worlds), its master link, or other servers (the
worlds on chat, every server on master, a world's chat link, which is
now counted too). HTTP requests carrying X-Darkflame-Server count as
another server's, the dashboard counts its own requests to the UGC
server, and each HTTP client address is counted. The report also gets
each RakNet connection's statistics (worlds name the player on it),
trimmed to the 32 busiest with the rest summed. Counting stays on the
main loop, except the dashboard's UGC fetches, which only touch the
locked recorder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:28:39 -05:00
Aaron Kimbrell
8a53562629 docs: chat histories and chat flags
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:23:23 -05:00
Aaron Kimbrell
6c43753fb2 feat(dashboard): chat flags queue
Staff flag chat for review (chat_flag): one message or several from one
conversation, with who it is about and a note. The flag keeps a copy of the
messages and 10 before and after. The Chat Flags page (Inbox, badge for open
flags) filters by status, character and account; a flag links to the character,
the account (mute, warn, ban), their player reports, their other flags and the
conversation, and shows its history. Reviewers (chat_flag_review) mark it
actioned or dismissed, change the note, link a player report or comment; all of
it is in the flag's history and the audit log. The sidebar adds Team Chat.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:23:23 -05:00
Aaron Kimbrell
f3a49991d4 feat(dashboard): guild, team and whisper chat histories
Guild chat per guild (/chat_log/guild/<id>, from the guild's card), team chat by
team (/chat_log/teams) and a character's whispers by conversation
(/characters/<id>/whispers, chat_dms; opening one is audited), paged on the
server. The Chat Log filters by account and time range, shows the conversation
around a message, links to its history and marks flagged messages; messages can
be picked (shift-click for a range) to flag. The character page links to all of
them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:23:23 -05:00
Aaron Kimbrell
6fd0dbd723 feat(dashboard): keep chat for 30 days by default
log_chat_days defaults to 30 (was 90). Pruning stays on the Log pruning task's
own database connection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:23:23 -05:00