TrafficStats keeps packets and bytes in and out per second and the busiest
packet and game message types. dServer counts at its send and receive calls
and adds RakNet's connection statistics (datagrams, resends, ping); the report
goes to master as SERVER_TRAFFIC (appended), which passes it to the dashboard.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new server (dUgcServer, started by master with enable_ugc_server=1) that
takes unprocessed ugc and ugc_modular_build rows from the database and makes
what the client downloads with UGCUSE3DSERVICES: an optimized NIF (hidden
faces removed, ambient occlusion baked into vertex colors) and a 128px DDS
icon for player models, rendered by a software rasterizer from the client's
LDD brick primitives, and icons for cars and rockets assembled from their
modules per ModularBuildComponent/ModuleComponent. It serves them, with the
models' LXFML, over HTTP in the client's UGCC<dc>/3DOPTIMIZED and
IMAGE128DDS layout with .gz and .checksum files, and keeps its folder under
a size cap.
Processing state lives in ugc.is_optimized plus new processed_at,
process_attempts and process_error columns (and the same on
ugc_modular_build). ServiceType::UGC is appended. NifFile moves to dCommon
and records named node transforms for the modules' attach points.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing in the server writes or reads a packet by hand any more, so the
helpers for doing so go:
- CBITSTREAM, CMSGHEADER, CINSTREAM, CINSTREAM_SKIP_HEADER, SEND_PACKET,
SEND_PACKET_BROADCAST and HEADER_SIZE leave dCommonVars.h;
- the free BitStreamUtils::WriteHeader (LUBitStream::WriteHeader writes the
same bytes) and the unused PacketUtils::SavePacket are deleted.
The last raw reads are replaced: WorldServer builds its input stream
directly, the master packet logs read the header with
LUBitStream::ReadHeader instead of peeking at packet->data[1] and [3], and
MessageInspector reads a sent game message's header with the new
NetGameMsg::ReadPacketHeader (the counterpart of WritePacket) instead of
memcmp/memcpy. packet->data[0] is still compared with RakNet's own
connection IDs.
The frozen oracles keep using the macros verbatim through the test-only
tests/dGameTests/LegacyPacketMacros.h; the HeaderSkip tests, which only
tested CINSTREAM_SKIP_HEADER, are removed.
docs/PacketArchitecture.md: "where we are" now describes the final state
and what still touches raw bytes (RakNet IDs, replica headers, behavior bit
streams), and a new section collects the known wire discrepancies found
during the conversion, with client addresses.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every MASTER service packet is now an LUBitStream struct (docs/PacketArchitecture.md,
PR 14) and the master server's switch is a dispatch map (PacketDispatcher), as are the
master handlers of the world, chat and dashboard servers.
- dNet/MasterPackets.h: RequestZoneTransfer, RequestZoneTransferResponse, ServerInfo,
RequestSessionKey, SetSessionKey, SessionKeyResponse, NewSessionAlert, PlayerAdded /
PlayerRemoved, CreatePrivateZone, RequestPrivateZone (passwords still cut to 50
characters when read), WorldReady, WorldReadyInfo (WORLD_READY to the dashboard),
PrepZone, Shutdown, ShutdownResponse, WorldShutDown (SHUTDOWN_RESPONSE to the
dashboard), ShutdownUniverse, AffirmTransferRequest/Response, RequestServerList,
ServerListResponse, DashboardShutdown, ConfigReload, InstanceShutdown. The Send*
functions are gone; MasterPackets::SendToMaster(msg) and SendTo(sysAddr, msg) send a
struct.
- The dashboard and instance migration structs (PlayerAction, DataChanged, Dashboard
messages, MessageCapture, InstanceMigration) are LUBitStreams of the MASTER service now
and moved to dNet/master/, included by MasterPackets.h. Their payloads are unchanged;
master forwards them by re-serializing the struct instead of copying raw bytes.
- InstanceManager, ZoneInstanceManager, MigrationCoordinator, dServer (server info, zone
transfer response), auth (SET_SESSION_KEY), the world (session keys, player added and
removed, world ready, shutdown response, affirmations, prep zone, shutdown universe) and
the dashboard (server list, instance shutdown, config reload, announcements, player
actions, message capture) send and read structs.
- The login stamps are a `stamps` field of RequestZoneTransfer and
RequestZoneTransferResponse (read leniently as before: a message without them reads as
empty); master adds its stamps in the REQUEST_ZONE_TRANSFER handler and when it answers,
as it did.
- InstanceManager::GetInstanceBySysAddr takes a const address.
Verified: tests/dGameTests/dNetTests/Legacy/MasterPacketsLegacy.h is a verbatim copy of
the old writers and readers; MasterPacketsTests requires identical bytes for a grid of
inputs, checks the old readers read what the structs write, round trips and truncation,
hand written golden packets, that zone transfers without stamps still read, that the dashboard/migration structs write what
"header + Serialize" wrote, and that the dispatcher drops truncated packets. No wire
bytes changed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The login's Stamps now go with auth's REQUEST_ZONE_TRANSFER to master and
come back in REQUEST_ZONE_TRANSFER_RESPONSE, so master stamps the steps it
performs itself, with its own time: got the request
(WORLD_PACKET_RECEIVED, zone), the world is still starting and the request
waits (IM_LOGIN_QUEUED, instance), answered with a world
(WORLD_SESSION_CONFIRM_TO_AUTH, instance). Auth sends what comes back in
the login response.
Server to server wire change (message IDs unchanged): both messages end
with a Stamps list (u32 16 * count + 4, then the stamps); it is empty
(4 bytes) when a world server asks for a transfer. Touched:
- MasterPackets::SendZoneTransferRequest / SendZoneTransferResponse: take
and write the stamps (default empty)
- MasterServer.cpp REQUEST_ZONE_TRANSFER: reads them, stamps, keeps them
in the PendingInstanceRequest (new stamps member, InstanceManager.h)
- InstanceManager::ReadyInstance / AffirmTransfer: stamp and send them back
- ZoneInstanceManager: HandleRequestZoneTransferResponse reads them; a
RequestZoneTransfer overload takes stamps and a callback that gets them
- AuthPackets LoginRequest::Handle uses that overload
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The NexusDashboard-parity dashboard (dDashboardServer) and everything built on it on the experimental branch:
accounts, characters, properties and moderation tools, permissions shared with in-game slash commands, economy
reports, World 3D and property 3D views with client scenery, scheduled events (features, vanity changes, live
events, announcements, restarts), vanity files and events, the CDClient browser, the message inspector with saved
captures, chat filter tools, community challenges, live ops, the AI moderator helper, and the server-side changes
they need.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat: logging structure improvements
tested that logs are sorted by the folder argument and that the directories are created recursively.
tested that crash dumps follow the same exact name (+Crash_...name..._pid.log) so you can match the crash dump to the corresponding log file much easier
* Potential fix for pull request finding 'Use parsed cloneID when constructing the log folder'
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* const
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix: bugs in private instances causing master crashes
tested that creating a private instance and shutting down the server no longer crashes master
* Update InstanceManager.cpp
* fix: security vulnerabilities
Tested that all functions related to the touched files work
will test sqlite on a CI build
* fix failing test
* ai feedback
* add buffer size checking
* use c_str
* dont log session key
* Try this for a mac definition
* be quiet apple
* fix: security fixes
dont print passwords for worlds
bound strings from clients
actually enable encryption between rakpeers
dont allow underflow when reading a string
Tested that packets are encrypted
tested that models can still be built
tested that combat still works
* add check
* use c++ nullptr instead of NULL
* initialize to 0
* globalize constant (should be in a namespace at least in the future)
* Update GameMessages.cpp
* check bounds
* feat: re-write persistent object ID tracker
Features:
- Remove random objectIDs entirely
- Replace random objectIDs with persistentIDs
- Remove the need to contact the MASTER server for a persistent ID
- Add persistent ID logic to WorldServers that use transactions to guarantee unique IDs no matter when they are generated
- Default character xml version to be the most recent one
Fixes:
- Return optional from GetModel (and check for nullopt where it may exist)
- Regenerate inventory item ids on first login to be unique item IDs (fixes all those random IDs
Pet IDs and subkeys are left alone and are assumed to be reserved (checks are there to prevent this)
There is also duplicate check logic in place for properties and UGC/Models
* Update comment and log
* fix: sqlite transaction bug
* fix colliding temp item ids
temp items should not be saved. would cause issues between worlds as experienced before this commit
fixes hardcore modes uscore drops
adds config option for excluded item drops
f
feat: Add logging for config options on load and reload
feat: Add logging for config options on load and reload
* merge ServerType and ServiceID enums
* rename eConnectionType to ServiceType in preparation for enum unification
* unify ServiceID and ServiceType enums
* shrink ServiceType to an 8-bit integer
* fix linux compilation error and update gamemsg test
* return to uint16_t
* Update dNet/AuthPackets.cpp
Use cast instead of padding
Co-authored-by: David Markowitz <39972741+EmosewaMC@users.noreply.github.com>
* Add default case to MasterServer.cpp
* move ref back to type
* Another formatting fix
* Fix comment to be more accurate
---------
Co-authored-by: jadebenn <9892985+jadebenn@users.noreply.github.com>
Co-authored-by: David Markowitz <39972741+EmosewaMC@users.noreply.github.com>
* simplify leaderboard code, fully abstract database
* update exception catching
* update exception catching and sql references, remove ugc from gamemessages
fix deleting model
remove unrelated changes
Update GameMessages.cpp
* remove ugc from gamemessages
* Update GameMessages.cpp
* Update Leaderboard.cpp
* bug fixes
* fix racing leaderboard
* remove extra stuff
* update
* add sqlite
* use a default for optimizations
* update sqlite
* Fix limits on update and delete
* fix bugs
* use definition to switch between databases
* add switch for different backends
* fix include guard and includes
* always build both
* add mysql if block
* Update Database.cpp
* add new options and add check to prevent overriding mysql
* correct config names
* Update README.md
* Update README.md
* merge to 1 sql file for sqlite database
* move to sqlite folder
* add back mysql migrations
* Update README.md
* add migration to correct the folder name or mysql
* yes aron
* updates
* Update CMakeLists.txt
* dont use paths at all, add where check to only update if folder name still exist
check also doesnt check for slashes and assumes one will be there since it will be.
* default dont auto create account
for releases we can change this flag
* default 0
* add times played query
* fix leaderboard not incrementing on a not better score
* add env vars with defaults for docker
* use an "enum"
* default to mariadb
* Update .env.example
* add admin account creation options from cli
* use actual gm levels
felt under delivered in previous iteration.
* Update dMasterServer/MasterServer.cpp
Co-authored-by: Daniel Seiler <me@xiphoseer.de>
---------
Co-authored-by: Daniel Seiler <me@xiphoseer.de>
Tested with logs that queries to get soft and hard cap actually succeed now
Logs about slash command handler command registration and vanity NPC creation in mis matched worlds are now removed.
* chore: supress warnings on external library headers and actually get rid of the last old-style casts
* remove commented out section I forgot
* update cmake required version to 3.25 unless we can find another way to do this
* update readme
* Update CMakeLists.txt
* Changed how the TryParse function works (and also did some general cleanup along the way)
* Update noexcept attributes (verified these are correct)
* Add fp overload for MacOS functionality
* resolving some feedback
* Split out unrelated changes to CleanupRoundup branch
* Update in response to feedback
* the consequences of emo's member variable renaming request
* Revert "the consequences of emo's member variable renaming request"
This reverts commit bf318caeda.
* Fully revert renaming attempt
* Revert "the consequences of emo's member variable renaming request"
This reverts commit bf318caeda.
Fully revert renaming attempt
* Created ClientVersion.h and moved the client version defaults to it
* Fix partial parsing and MacOS floating point errors
* attempting fix to MacOS compiler error
* syntax pass (should be the last commit unless the CI fails)
* ah, wait, forgot to uncomment the preprocessor statements for MacOS. THIS should be the last commit pending CI
* Okay, one last thing I noticed: We were including C headers here. Now they're C++ headers. Pinky swear this is it!
* typo and I am OCD. please let this be the last
* hash is usally but not always noexcept, so the specifier should go
* Address MOST of the feedback
* address the claim codes issue
* chore: continue work on removing raw packet reading
tested that logging in, deleted a char, renaming a char, and transfeering to a zone all work still
* Address Feedback
* Logger to Server class
Dont handle master packets from our clients
* move to namespace
Revert "remove extra headers"
This reverts commit ac7b901ece22165cdb0f38fca4be7d8fdf004de8.
remove extra headers
no changes otherwise.
* Merge branch 'main' into server_consolidation_of_work
* Update WorldServer.cpp
* fix submodule version
---------
Co-authored-by: Aaron Kimbre <aronwk.aaron@gmail.com>
* greatly simplify docker
* back to gcc since I can't the libs to load properly
* fix typo
add set -e to entrypoint
better copy of entrypoint.sh and use proper entrypoint
* use debian instead of gcc for runtime
comment and organize it a bit
drop gcc to 12 since we are using debian 12 as well
* explicitly include mariadb libs
* Make the server not crash in the case we are using only env-vars
make the dockerfile have configs in the expected location incase of bypassing entrypoint.sh
* remove unneede var from example, since it's in the container now
* coments to dockerfile
* Revert master server changes
* Resolve conflicting port options between chat, master, and world
move chat_server_port to shared since it's used by world and chat
* Don't error if file does not exists when updating a config option
move update before and use bin dir var
* chore: organize build flags
* Remove ambiguous include path
Don't be default incluyde bcrypt so you need to specify the folder. Allows pre-processor to find the correct file.
* Revert settings
* working
f
* Remove std::couts littered throughout the base
* working
End of optimizations for now
going faster
* Remove extraneous compare function
std::less<LWOSCENEID> already does this in a map.
* gaming
* Update Zone.cpp
* dlu is moving to bitbucket again
* Update Level.cpp
---------
Co-authored-by: Jettford <mrjettbradford@gmail.com>
* Database: Convert to proper namespace
* Database: Use base class and getter
* Database: Move files around
* Database: Add property Management query
Database: Move over user queries
Tested at gm 0 that pre-approved names are pre-approved, unapproved need moderator approval
deleting characters deletes the selcted one
refreshing the character page shows the last character you logged in as
tested all my characters show up when i login
tested that you can delete all 4 characters and the correct character is selected each time
tested renaming, approving names as gm0
Database: Add ugc model getter
Hey it works, look I got around the mariadb issue.
Database: Add queries
Database: consolidate name query
Database: Add friends list query
Update name of approved names query
Documentation
Database: Add name check
Database: Add BFF Query
Database: Move BFF Setter
Database: Move new friend query
Database: Add remove friend queries
Database: Add activity log
Database: Add ugc & prop content removal
Database: Add model update
Database: Add migration queries
Database: Add character and xml queries
Database: Add user queries
Untested, but compiling code
Need to test that new character names are properly assigned in the following scenarios
gm 0 and pre-approved name
gm 0 and unapproved name
gm 9 and pre-approved name
gm 9 and unapproved name
Database: constify function arguments
Database: Add pet queries
* Database: Move property model queries
Untested. Need to test
placing a new model
moving existing one
removing ugc model
placing ugc model
moving ugc model(?)
changing privacy option variously
change description and name
approve property
can properly travel to property
* Property: Move stale reference deletion
* Database: Move performance update query
* Database: Add bug report query
* Database: Add cheat detection query
* Database: Add mail send query
* Untested code
need to test mailing from slash command, from all users of SendMail, getting bbb of a property and sending messages to bffs
* Update CDComponentsRegistryTable.h
Database: Rename and add further comments
Datavbase: Add comments
Add some comments
Build: Fix PCH directories
Database: Fix time
thanks apple
Database: Fix compiler warnings
Overload destructor
Define specialty for time_t
Use string instead of string_view for temp empty string
Update CDTable.h
Property: Update queries to use mapId
Database: Reorganize
Reorganize into CDClient folder and GameDatabase folder for clearer meanings and file structure
Folders: Rename to GameDatabase
MySQL: Remove MySQL Specifier from table
Database: Move Tables to Interfaces
Database: Reorder functions in header
Database: Simplify property queries
Database: Remove unused queries
Remove extra query definitions as well
Database: Consolidate User getters
Database: Comment logs
Update MySQLDatabase.cpp
Database: Use generic code
Playkey: Fix bad optional access
Database: Move stuff around
WorldServer: Update queries
Ugc reduced by many scopes
use new queries
very fast
tested that ugc still loads
Database: Add auth queries
I tested that only the correct password can sign into an account.
Tested that disabled playkeys do not allow the user to play the game
Database: Add donation query
Database: add objectId queries
Database: Add master queries
Database: Fix mis-named function
Database: Add slash command queries
Mail: Fix itemId type
CharFilter: Use new query
ObjectID: Remove duplicate code
SlashCommand: Update query with function
Database: Add mail queries
Ugc: Fix issues with saving models
Resolve large scope blocks as well
* Database: Add debug try catch rethrow macro
* General fixes
* fix play key not working
* Further fixes
---------
Co-authored-by: Aaron Kimbre <aronwk.aaron@gmail.com>
