feat(capture): record whole packets of an account, a character or everything on every server

Staff arm a packet capture on the dashboard; master passes MESSAGE_CAPTURE_CONTROL ARM to
every world, auth and chat and arms its own. Each server's PacketCapture tap (dServer receive,
and a send hook in RakPeer::Send so replica constructions are seen too) records into one
preallocated chunk per server and ships sealed chunks through master on the main loop when
capture_flush_bytes or capture_flush_interval_ms is reached; past capture_buffer_max_mb the
oldest chunks are dropped and the dashboard records a gap. Nothing is armed: one flag check.

- targets: an account (from its login; packets before the login are kept per connection
  and added once auth or the world knows whose they are), a character (from when it is
  picked), or everything; up to 8 at once (a bit each in the record mask)
- worlds and auth record their clients' packets and the master link messages of a captured
  player (session keys by name, zone transfers by request, player added/removed, migration);
  chat finds the player in each packet; master records server traffic for everything
- secrets are never recorded: structs that carry them (login request, login response user
  key, world validation session key, session key messages between servers) are read,
  blanked and written again before recording; auth keeps only the handshake and login
- PacketDecoder: a registry by service and message id names every packet and decodes the
  registered structs; CaptureBundle is the file format (DLUBNDL1, metadata, records);
  CaptureTools orders records on one timeline, pulls movement out, makes bundles portable
  or anonymous and diffs replays
- the dashboard keeps packet captures in message_capture_sessions (capture_kind 1) and
  their packets in a file under capture_dir, one write per batch; arming is audited
- MESSAGE_CAPTURE_CONTROL/DATA only gain appended enum values and trailing fields

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Aaron Kimbrell
2026-09-27 08:45:12 -05:00
parent fb6d73e4bd
commit 332bc04ce8
35 changed files with 3207 additions and 15 deletions

74
dNet/CaptureTools.h Normal file
View File

@@ -0,0 +1,74 @@
#ifndef __CAPTURETOOLS__H__
#define __CAPTURETOOLS__H__
#include <cstdint>
#include <map>
#include <optional>
#include <string>
#include <vector>
#include "CaptureBundle.h"
#include "dCommonVars.h"
#include "json.hpp"
/**
* What the dashboard's capture viewer and the capture tool do with recorded packets (docs/CaptureReplay.md):
* describe them, order them on one timeline, pull the player's movement out, make a bundle portable or anonymous,
* and compare a replay's answers with the recorded ones. Pure functions over records, so they are unit tested.
*/
namespace CaptureTools {
// The records of all servers on one timeline: by time, then by server and its sequence
void SortTimeline(std::vector<CaptureBundle::Record>& records);
// Whether a record went from a game client to a server
bool FromClient(const PacketRecordHeader& header);
// One record for the viewer: where it went, its name and, when `fields`, its decoded fields
nlohmann::json RecordJson(const CaptureBundle::Record& record, size_t index, int64_t startUs, bool fields);
struct Track {
LWOOBJID characterId{};
uint32_t zoneId{};
uint32_t instanceId{};
std::vector<float> samples; // t (seconds from the capture's start), x, y, z, ...
};
// Where each captured character moved (their POSITION_UPDATEs), per zone and instance
std::vector<Track> Tracks(const std::vector<CaptureBundle::Record>& records, int64_t startUs);
/**
* Makes a bundle portable: the source server's character and account IDs are replaced by placeholders
* (PLACEHOLDER_BASE + n, written in the records' bytes and headers), listed in meta.ids as "char#n" / "account#n";
* account names and session fields are already blank (they are never recorded). Returns the characters found, by
* symbol, with their source ID, for the setup section.
*/
constexpr int64_t PLACEHOLDER_BASE = 0x1FEDC00000000000LL;
std::map<std::string, LWOOBJID> MakePortable(CaptureBundle::Bundle& bundle);
// Blanks what players typed and names (chat text, character and account names) in every packet whose struct is
// known, so a bundle can be kept as a test fixture. Returns how many packets were changed.
size_t Anonymise(CaptureBundle::Bundle& bundle);
/**
* A replay's answers against the recorded ones. Server->client packets are paired in order by name; paired
* packets are compared by their decoded fields, leaving out what legitimately differs between runs (object IDs
* the server makes, timestamps, session keys, instance and clone IDs, server addresses).
*/
struct DiffReport {
size_t expected{};
size_t matched{}; // same fields
size_t differing{}; // same packet, different fields
size_t missing{}; // recorded, not answered in the replay
size_t extra{}; // answered in the replay, not recorded
std::map<std::string, size_t> differingByName;
std::map<std::string, size_t> missingByName;
std::map<std::string, size_t> extraByName;
std::vector<std::string> examples; // the first few differences, readable
nlohmann::json ToJson() const;
};
DiffReport Diff(const std::vector<CaptureBundle::Record>& expected, const std::vector<CaptureBundle::Record>& actual);
// Fields left out of comparisons (by name, in any packet)
bool IsVolatileField(const std::string& name);
}
#endif //!__CAPTURETOOLS__H__