ZoneFileLog keeps the .luz, .lvl, .lutriggers, terrain .raw and navmesh a
world read while loading its zone: absolute path of the loose file (or
the name in the pack), size and an FNV-1a hash of the bytes read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FdbSnapshot names copies cdclient-<hash>.fdb / CDServer-<hash>.sqlite, writes
them under temporary names and renames them in, keeps a pointer file naming
the current pair, removes old copies, and decides when a changed file has
settled enough to hash. Nothing in it logs, so it can run on a worker.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Profiler.h: frames, named scopes and phases, recorded on the main thread only (other threads' scopes do nothing). Per second: frames, total and longest frame time, a mergeable frame time histogram (TrafficStats::Histogram) and time per phase. Per report: the packet types that took longest, the worst frames and the frames over a threshold with their heaviest scopes. Profiling sessions merge every frame's scope tree for a while into one (folded stacks for flame graphs). Optional Tracy client (DLU_TRACY, off by default) gets the same frames and scopes. Task 96.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FdbMappedFile maps a file read-only (CreateFileMapping/MapViewOfFile on
Windows, mmap elsewhere) and falls back to reading it into memory when
mapping fails. FdbReader reads the table and column headers from it and
looks rows up by their first column through the fdb's own buckets,
decoding every integer as little-endian with bounds checks, so the rows
never get copied out of the file.
Tests write small fdb files (collisions, text, int64, nulls) and read
them both mapped and from memory.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The login request's memoryStats is a run of short parts with no separators
(process working set and private bytes, memory load, physical memory, commit
limit, the 32-bit client's own address space, peaks). ParseMemoryStats splits
it into numbers; a text cut short keeps the parts before the cut.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cmake/BuildInfo.cmake runs on every build and rewrites the generated
BuildInfo.cpp only when something changed, so a new commit recompiles one
file and relinks. The build kind comes from DLU_BUILD_KIND
(release/ci/local) or CI=true, defaulting to local.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What someone may do on the dashboard is now their GM level's permissions plus the grants on their account, minus its
denies (PermissionGrants.h). A deny beats a grant; denies never apply to GM 9, and settings and permissions_manage stay
GM 9 only. The account's grants are read with every request (like its GM level), so a change applies at once, and
they are passed through every check: RouteUtils::Can, CanViewCharacter, the rank rules (self_* and manage_equal_rank),
routes guarded by a permission, the templates' `can`, the API documentation, API access, API key scopes (a key never
does more than its owner may now) and WebSocket subscriptions.
New permission grants_manage (GM 9 by default) and the API to manage grants: GET /api/grants/catalog, GET /api/grants,
POST /api/grants, POST /api/grants/:id/remove. Nobody grants or takes away what they don't hold themselves (a
permission, every permission of a group, a command they may use, every command up to their own GM level), and only on
accounts the rank rules let them manage (their own with self_moderation). Commands with a fixed level or a floor
above GM 1 (/execute) can't be granted. Every change goes in the audit log (grant_permission, deny_permission,
remove_grant). Also: the Showcase gate and the traffic subscription now check their permission by name.
Check: grant a GM 2 account accounts_ban (it can ban, and the Ban button shows); deny a GM 8 account accounts_view (the
accounts list is refused); give an expiry a minute ahead and see it stop; try to grant a permission your account
doesn't have (refused); dWebTests PermissionGrantsTests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The client keeps the scene under the player loaded, the scenes the zone
file's transitions connect to it, and the global scene
(Zone::StreamScenesAroundPosition 0x0108a3f0, TerrainManager::GetSceneAtPos
0x01069010, the connected scenes at 0x01066500; transitions naming a
missing scene dropped as Zone::FixupInvalidTransitions 0x010842e0 does).
- ZoneScenes (dCommon): the terrain's scene map lookup and the scene graph,
shared by the world server and the dashboard.
- Objects remember the scene they were placed in (spawners pass theirs on).
- ghosting_scenes=1 (world config, off by default): players get the objects
of their loaded scenes instead of the ones within the ghosting distances;
objects from no scene go by the scene under them. Zones without a scene
map keep distance ghosting.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TrafficStats keeps packets and bytes in and out per second and the busiest
packet and game message types. dServer counts at its send and receive calls
and adds RakNet's connection statistics (datagrams, resends, ping); the report
goes to master as SERVER_TRAFFIC (appended), which passes it to the dashboard.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new server (dUgcServer, started by master with enable_ugc_server=1) that
takes unprocessed ugc and ugc_modular_build rows from the database and makes
what the client downloads with UGCUSE3DSERVICES: an optimized NIF (hidden
faces removed, ambient occlusion baked into vertex colors) and a 128px DDS
icon for player models, rendered by a software rasterizer from the client's
LDD brick primitives, and icons for cars and rockets assembled from their
modules per ModularBuildComponent/ModuleComponent. It serves them, with the
models' LXFML, over HTTP in the client's UGCC<dc>/3DOPTIMIZED and
IMAGE128DDS layout with .gz and .checksum files, and keeps its folder under
a size cap.
Processing state lives in ugc.is_optimized plus new processed_at,
process_attempts and process_error columns (and the same on
ugc_modular_build). ServiceType::UGC is appended. NifFile moves to dCommon
and records named node transforms for the modules' attach points.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The NexusDashboard-parity dashboard (dDashboardServer) and everything built on it on the experimental branch:
accounts, characters, properties and moderation tools, permissions shared with in-game slash commands, economy
reports, World 3D and property 3D views with client scenery, scheduled events (features, vanity changes, live
events, announcements, restarts), vanity files and events, the CDClient browser, the message inspector with saved
captures, chat filter tools, community challenges, live ops, the AI moderator helper, and the server-side changes
they need.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* change network settings from vector to LwoNameValue
* move settings on Entity to managed memory
* Migrate more members
* chore: remove pointer leakage from raw ldf pointers
* feedback
* fix ci
Tested that models are migrated to the new format a-ok
Tested that the new logic works as expected.
Old code needs to be kept so that models in both states can be brought to modern standards
* Break out changes into a smaller subset
* NL@EOF
* fix windows bs
add player ws updates
add websocket docs
* tested everything to make sure it works
* Address Feedback
* Add utilities for formats
* Normalize model positions when placing in the world
Have tested that placing a small and very large model both place and are located at the correct position.
* add migration
* Update Logger.cpp
* add some notes and remove some logs
* change arguments and add eof check
Revert "fix: buff station cycling and dying too soon"
This reverts commit 1c6cb2921e10eb2000ac40007d0c2636ba2ac151.
fix: buff station cycling and dying too soon
Tested that the buff station now only cycles after it has been built and has been alive for 25 seconds.
* simplify leaderboard code, fully abstract database
* update exception catching
* update exception catching and sql references, remove ugc from gamemessages
fix deleting model
remove unrelated changes
Update GameMessages.cpp
* remove ugc from gamemessages
* Update GameMessages.cpp
* Update Leaderboard.cpp
* bug fixes
* fix racing leaderboard
* remove extra stuff
* update
* add sqlite
* use a default for optimizations
* update sqlite
* Fix limits on update and delete
* fix bugs
* use definition to switch between databases
* add switch for different backends
* fix include guard and includes
* always build both
* add mysql if block
* Update Database.cpp
* add new options and add check to prevent overriding mysql
* correct config names
* Update README.md
* Update README.md
* merge to 1 sql file for sqlite database
* move to sqlite folder
* add back mysql migrations
* Update README.md
* add migration to correct the folder name or mysql
* yes aron
* updates
* Update CMakeLists.txt
* dont use paths at all, add where check to only update if folder name still exist
check also doesnt check for slashes and assumes one will be there since it will be.
* default dont auto create account
for releases we can change this flag
* default 0
* add times played query
* fix leaderboard not incrementing on a not better score
* add env vars with defaults for docker
* use an "enum"
* default to mariadb
* Update .env.example
* fix: more include changes
* fix: remove dZoneManager from global include
* fix: dDatabase
* fix: dCommon
* fix: object libs
* fix: rebase
* fix: bcrypt
* wip: try simplified connector build
* fix: update dockerfile
* fix: mariadb C/C++ on apple
* feat: Move scripts to CMAKE_MODULE_PATH
* fix: dPropertyBehaviors
* fix: macos?
* fix: Dockerfile
* fix: macos?
* fix: macos?
* fix: macos?
* fix: macos?
* fix: macos?
* try: install_name_tool
* fix not building on unix
* fix include paths
* Remove code changes
Will fix in another PR.
* format pass
remove 2 more included directories.
remove commented out code
add status to messages
* comments and format
surround include directories with quotes
remove commented out code
remove debug messages
* Update CMakeLists.txt
---------
Co-authored-by: David Markowitz <EmosewaMC@gmail.com>
Co-authored-by: David Markowitz <39972741+EmosewaMC@users.noreply.github.com>
* Update PlayerContainer.cpp
fix: turn warnings into errors
for the few warnings we get, at least make sure we listen to them now on unix platforms. Windows has too many right now to enable /WX
resolve warning
actually fix it
Update CMakeLists.txt
* detour pls
* Update CMakeLists.txt
* I HAVE 20 DOLLARS AND I NEED A WII GAME FOR MY KID
* I HAVE 0 DOLLARS NOW
* don't look don't look
* Revert "don't look don't look"
This reverts commit 5603eb5980.
* Revert "Revert "don't look don't look""
This reverts commit a334832a4d.
* could it be
* we found one (but its already reported)
not resolved yet though.
* Revert "don't look don't look"
This reverts commit 5603eb5980.
* ignore warning for file
* another one
* Update .gitmodules
* comments
* Logger: Rename logger to Logger from dLogger
* Logger: Add compile time filename
Fix include issues
Add writers
Add macros
Add macro to force compilation
* Logger: Replace calls with macros
Allows for filename and line number to be logged
* Logger: Add comments
and remove extra define
Logger: Replace with unique_ptr
also flush console at exit. regular file writer should be flushed on file close.
Logger: Remove constexpr on variable
* Logger: Simplify code
* Update Logger.cpp
* Update AMFDeserializeTests.cpp
Redo Amf3 functionality
Overhaul the whole thing due to it being outdated and clunky to use
Sometimes you want to keep the value
Update AMFDeserializeTests.cpp
* Fix enum and constructors
Correct enum to a class and simplify names.
Add a proper default constructor
* Update MasterServer.cpp
* Fix bugs and add more tests
* Refactor: AMF with templates in mind
- Remove hard coded bodge
- Use templates and generics to allow for much looser typing and strengthened implementation
- Move code into header only implementation for portability
Refactor: Convert AMF implementation to templates
- Rip out previous implementation
- Remove all extraneous terminology
- Add proper overloads for all types of inserts
- Fix up tests and codebase
* Fix compiler errors
* Check for null first
* Add specialization for const char*
* Update tests for new template specialization
* Switch BitStream to use references
* Rename files
* Check enum bounds on deserialize
I did this on a phone
Allows the server to be run from a non-build directory. Also only read or write files relative to the build directory, regardless of where the server is run from
* First iteration of pack reader and interface
* Fix memory leak and remove logs
* Complete packed asset interface and begin on file loading replacement
* Implement proper BinaryIO error
* Improve AssetMemoryBuffer for reading and implement more reading
* Repair more file loading code and improve how navmeshes are loaded
* Missing checks implementation
* Revert addition of Manifest class and migration changes
* Resolved all feedback.
* Properly store BBB in database
Store the BBB data in the database as the received SD0 packet as opposed to just the raw lxfml. Addressed several memory leaks as well.
* Add Sd0Conversion
Add brick by brick conversion commands with 2 parameters to tell the program what to do with the data.
Add zlib -> sd0 conversion. Files look good at a glance but should be tested in game to ensure stability. Tests to come.
* moving to laptop
ignore this commit. I need to move this to my laptop
* Add functionality to delete bad models
Adds functionality to delete bad models. Models are batched together and deleted in one commit.
More testing is needed to ensure data safety. Positive tests on a live database reveal the broken models were truncated and complete ones were kept around successfully. Tests should be done to ensure larger sd0 models are properly saved and not truncated since this command should be able to be run any time.
Valgrind tests need to be run as well to ensure no memory leaks exist.
* Delete from query change
Changed from delete to delete cascade and instead deleting from properties_contents as opposed to ugc.
* Address numerous bugs
DELETE CASCADE is not a valid SQL command so this was changed to a better delete statement.
Added user confirmation before deleting a broken model.
Address appending the string model appending bad data, causing excess deletion.
Addressed memory leaks with sql::Blob
* Error handling for string
* Even more proper handling...
* Add bounds check for cli command
Output a message if a bad command is used.
Update MasterServer.cpp
* Remove user interference
-Add back in mariadb build jobs so i dont nuke others systems
- Remove all user interference and consolidate work into one command since 1 depends on the next.
* Add comments
test
Revert "test"
This reverts commit fb831f268b7a2f0ccd20595aff64902ab4f4b4ee.
* Update CMakeMariaDBLists.txt
Test
* Improve migration runner
Migration runner now runs automatically.
- Resolved an issue where extremely large sql queries caused the database to go into an invalid state.
- Made migrations run automatically on server start.
- Resolved a tiny memory leak in migration runner? (discarded returned pointer)
- Moved sd0 migrations of brick models to be run automatically with migration runner.
- Created dummy file to tell when brick migrations have been run.
* Update README
Updated the README to reflect the new server migration state.
* Make model deleter actually delete models
My complicated sql actually did nothing... Tested that this new SQL properly gets rid of bad data.
* Revert "Update CMakeMariaDBLists.txt"
This reverts commit 8b859d8529.
* Use RecastNavigation CMakeLists
* Use tinyxml2 CMakeLists
* Use bcrypt CMakeLists
* Move variable init to CMakeLists
This has to be done here to prevent missing dependency errors.
* General improvements
Only link dynamic if on gnu
use more thirdparty cmakes
* Disable tinyxml2 testing
* Add AMFDeserializer
Add an AMFDeserializer
Reverted unrelated changes
Add unit tests for AMFDeserializer
Added unit tests for the AMFDeserializer
Finish tests
Finish the AMF deserializer tests. This commit finishes the positive test case and implements a load test case that is expected to take less than 1.5 seconds to process.
Modularized tests
Made tests a bit modular and split into more methods
Specified binary read from file
Specified that on the IO stream we are reading a binary file otherwise windows will terminate reading the binary file on seeing a 1A byte.
Added more tests
Added tests for unimplemented values and edited a test file to be more modular
Updated test text
Fix spacing
Update AMFDeserializeTests.cpp
* Update CMakeLists.txt
* Update AMFDeserializeTests.cpp
f
Actually follow the AMF spec
Update AMFDeserializeTests.cpp
tabs
Add in commented tests
* Follow spec
formatting
Add Integer Tests
Follow Spec more
Follow spec
* Use unique_ptr
* Update AMFDeserialize.cpp
Semantics
Update AMFDeserialize.cpp
Add new lines to EOF
CMake fix
* Add better std string read
Co-authored-by: Daniel Seiler <xiphoseer@mailbox.org>
* make not static
Co-authored-by: Daniel Seiler <xiphoseer@mailbox.org>
* Update CMake configuration for easier maintenance
* Incorrect casing fix
* Move package requirement
* Update CTest linking
* Add logs to the CMake
* Add linking for common libraries
Added linking for common libraries in tests subdirectory.
* Move test subdirectory higher up for some reason
* Whitespace a log removal
Missed new line
* Add dCommon to dChatFilter
* Update library output dir
* Correct libBcrypt
* Further refactor CMake behaviour
* Repair bad comments and update library defines
* Revert to old include directory method
* Implement platform defines
* Add missing include
Mac needs a specific include for defining platform. Does not compile without this.
Co-authored-by: EmosewaMC <39972741+EmosewaMC@users.noreply.github.com>
