Commit Graph

71 Commits

Author SHA1 Message Date
David Markowitz
e58218cfbc
chore: Speed up and cleanup level and zone loading; Add safer asset buffer reading (#1314)
* Remove std::couts littered throughout the base

* working

End of optimizations for now

going faster

* Remove extraneous compare function

std::less<LWOSCENEID> already does this in a map.

* gaming

* Update Zone.cpp

* dlu is moving to bitbucket again

* Update Level.cpp

---------

Co-authored-by: Jettford <mrjettbradford@gmail.com>
2023-12-23 11:24:16 -06:00
David Markowitz
7f623d358c
refactor: Database abstraction and organization of files (#1274)
* Database: Convert to proper namespace

* Database: Use base class and getter

* Database: Move files around

* Database: Add property Management query

Database: Move over user queries

Tested at gm 0 that pre-approved names are pre-approved, unapproved need moderator approval
deleting characters deletes the selcted one
refreshing the character page shows the last character you logged in as
tested all my characters show up when i login
tested that you can delete all 4 characters and the correct character is selected each time
tested renaming, approving names as gm0

Database: Add ugc model getter

Hey it works, look I got around the mariadb issue.

Database: Add queries

Database: consolidate name query

Database: Add friends list query

Update name of approved names query

Documentation

Database: Add name check

Database: Add BFF Query

Database: Move BFF Setter

Database: Move new friend query

Database: Add remove friend queries

Database: Add activity log

Database: Add ugc & prop content removal

Database: Add model update

Database: Add migration queries

Database: Add character and xml queries

Database: Add user queries

Untested, but compiling code

Need to test that new character names are properly assigned in the following scenarios
gm 0 and pre-approved name
gm 0 and unapproved name
gm 9 and pre-approved name
gm 9 and unapproved name

Database: constify function arguments

Database: Add pet queries

* Database: Move property model queries

Untested.  Need to test
placing a new model
moving existing one
removing ugc model
placing ugc model
moving ugc model(?)
changing privacy option variously
change description and name
approve property
can properly travel to property

* Property: Move stale reference deletion

* Database: Move performance update query

* Database: Add bug report query

* Database: Add cheat detection query

* Database: Add mail send query

* Untested code

need to test mailing from slash command, from all users of SendMail, getting bbb of a property and sending messages to bffs

* Update CDComponentsRegistryTable.h

Database: Rename and add further comments

Datavbase: Add comments

Add some comments

Build: Fix PCH directories

Database: Fix time

thanks apple

Database: Fix compiler warnings

Overload destructor
Define specialty for time_t
Use string instead of string_view for temp empty string

Update CDTable.h

Property: Update queries to use mapId

Database: Reorganize

Reorganize into CDClient folder and GameDatabase folder for clearer meanings and file structure

Folders: Rename to GameDatabase

MySQL: Remove MySQL Specifier from table

Database: Move Tables to Interfaces

Database: Reorder functions in header

Database: Simplify property queries

Database: Remove unused queries

Remove extra query definitions as well

Database: Consolidate User getters

Database: Comment logs

Update MySQLDatabase.cpp

Database: Use generic code

Playkey: Fix bad optional access

Database: Move stuff around

WorldServer: Update queries

Ugc reduced by many scopes
use new queries
very fast
tested that ugc still loads

Database: Add auth queries

I tested that only the correct password can sign into an account.
Tested that disabled playkeys do not allow the user to play the game

Database: Add donation query

Database: add objectId queries

Database: Add master queries

Database: Fix mis-named function

Database: Add slash command queries

Mail: Fix itemId type

CharFilter: Use new query

ObjectID: Remove duplicate code

SlashCommand: Update query with function

Database: Add mail queries

Ugc: Fix issues with saving models

Resolve large scope blocks as well

* Database: Add debug try catch rethrow macro

* General fixes

* fix play key not working

* Further fixes

---------

Co-authored-by: Aaron Kimbre <aronwk.aaron@gmail.com>
2023-11-17 18:47:18 -06:00
David Markowitz
d1dc9f5403
fix: add Onedrive log message (#1269)
* fix: Onedrive log message

* Update README.md
2023-11-08 17:56:59 -08:00
David Markowitz
5942182486
feat: Abstract Logger and simplify code (#1207)
* Logger: Rename logger to Logger from dLogger

* Logger: Add compile time filename

Fix include issues
Add writers
Add macros
Add macro to force compilation

* Logger: Replace calls with macros

Allows for filename and line number to be logged

* Logger: Add comments

and remove extra define

Logger: Replace with unique_ptr

also flush console at exit. regular file writer should be flushed on file close.

Logger: Remove constexpr on variable

* Logger: Simplify code

* Update Logger.cpp
2023-10-21 16:31:55 -07:00
David Markowitz
2746683235 Master: Address race condition
Fix a race condition that would overwrite the port of zone 0 with the auth server port or a chat server port which would cause a domino effect of desynced server ports

Tested that a sleep(5) in auth still allows a player to connect and login without issues
2023-10-14 00:01:41 -07:00
Aaron Kimbrell
08020cd86d
chore: cleanup LU(W)string writing (#1188)
* chore: cleanup LU(W)string writing
and add methods for reading
remove redunent "packet" from packet reading helpers
move write header to bitstreamutils since it's not packet related
add tests for reading/writing LU(W)Strings

* remove un-needed function defintions in header

* make reading and writing more efficient

* p

p

* quotes

* remove unneeded default

---------

Co-authored-by: David Markowitz <39972741+EmosewaMC@users.noreply.github.com>
2023-09-20 20:06:28 -05:00
David Markowitz
e5b69745aa
feat: Remove NET_VERSION define (#1153)
* Remove NET_VERSION define

I get to test this on 4 different platforms.  yay...

* move stuff around

finally fixed you docker

i love this

Update Dockerfile

Update Dockerfile

change type

Update AuthPackets.cpp

Update AuthServer.cpp

* Update CMakeVariables.txt

* Update sharedconfig.ini
2023-08-03 21:38:33 -05:00
David Markowitz
8a065ad074 Merge remote-tracking branch 'upstream/main' into more-cdclient-cleanup 2023-05-13 16:16:58 -07:00
David Markowitz
4fe335cc66
Refactor: Amf3 implementation (#998)
* Update AMFDeserializeTests.cpp

Redo Amf3 functionality

Overhaul the whole thing due to it being outdated and clunky to use

Sometimes you want to keep the value

Update AMFDeserializeTests.cpp

* Fix enum and constructors

Correct enum to a class and simplify names.
Add a proper default constructor

* Update MasterServer.cpp

* Fix bugs and add more tests

* Refactor: AMF with templates in mind

- Remove hard coded bodge
- Use templates and generics to allow for much looser typing and strengthened implementation
- Move code into header only implementation for portability

Refactor: Convert AMF implementation to templates

- Rip out previous implementation
- Remove all extraneous terminology
- Add proper overloads for all types of inserts
- Fix up tests and codebase

* Fix compiler errors

* Check for null first

* Add specialization for const char*

* Update tests for new template specialization

* Switch BitStream to use references

* Rename files

* Check enum bounds on deserialize

I did this on a phone
2023-05-13 17:22:00 -05:00
David Markowitz
33c12f3bc5
Fix bounds check (#1071)
Fix Chat Crash

Update CMakeVariables.txt

Add checks for all servers
2023-05-06 13:32:53 -05:00
Aaron Kimbrell
e297aacc68
Breakout message identifiers (#1065)
and make them scope enums
2023-05-03 16:38:32 -05:00
David Markowitz
426bc963fe Add Animation Table logic 2023-03-26 05:18:45 -07:00
David Markowitz
b432a3f5da Remove inlines
Clean up macros

more tomorrow

Cleanup and optimize CDActivities table

Remove unused include

Further work on CDActivityRewards

Update MasterServer.cpp

Further animations work

Activities still needs work for a better PK.

fix type

All of these replacements worked

Create internal interface for animations

Allows for user to just call GetAnimationTIme or PlayAnimation rather than passing in arbitrary true false statements
2023-03-26 02:59:46 -07:00
David Markowitz
7671cc6865
CDClient cleanup and optimization (#1023)
* CDClient cleanup and optimization

- Use static function to get table name
- Remove unused GetName function
- Replace above function with a static GetTableName function
- Remove verbose comments
- Remove verbose initializers
- Remove need to specify table name when getting a table by name
- Remove unused typedef for mac and linux

* Re-add unused table

Convert tables to singletons

- Convert all CDClient tables to singletons
- Move Singleton.h to dCommon
- Reduce header clutter in CDClientManager
2023-03-17 07:36:21 -07:00
Aaron Kimbrell
9d65d871d0
fix compatibility with nexus dash (#1011)
* add some more utility to the masterserver -a command
fix compatability with nexus dash

* cleanup code some more
user logger where it makes sense
only take in password if it is needed
log a better error if account cannot be created

* update message

* return before success statement if catching error
2023-03-05 13:11:32 -06:00
David Markowitz
fbaf1cbb25
Dont preemptively delete objects (#958)
Let finalizeShutdown take care of it
2023-01-17 06:06:09 -06:00
David Markowitz
bad3845d83
Address Docker issues and remove need to extract cdclient.fdb (#895)
* Implement a server res directory

* Only convert if neither exist

* Remove unzip, Update RegEx

* readme updates

Run setup after setting working dir

Address several docker issues

Revert "Run setup after setting working dir"

This reverts commit fd2fb9228e82a350204c1ef61f7ba059479bb12f.

Fix docker

* Remove extra submodules

* Rework logic

* Switch if block

* Remove need to extract fdb from client

* Change log name

* Update FdbToSqlite.cpp
2023-01-06 23:04:20 -06:00
David Markowitz
fd9757d121
Implement a server res directory for server required client files (#891) 2022-12-21 22:34:11 -08:00
David Markowitz
9f47b1f0bb Merge remote-tracking branch 'upstream/main' into MoreImprovements 2022-12-19 03:53:51 -08:00
Gie "Max" Vanommeslaeghe
f41dfaebdf
Merge pull request #887 from EmosewaMC/ServerShutdown
Improve server shutdown process
2022-12-18 15:41:58 +01:00
Jett
1da2db9db6 Resolve some string related issues for Windows Debug 2022-12-17 16:03:18 +00:00
David Markowitz
3f1b4339f5 Improve chat and Auth
Also change most uses of int to specified lengths.
2022-12-16 02:24:02 -08:00
EmosewaMC
3c581fffbb Remove magic numbers
Replace magic numbers with constexpr calculated times.

Tested that trying to create a new instance while shutting down doesn't allow a new instance to be created.
2022-12-15 20:39:29 -08:00
EmosewaMC
4775dbf27f Condense frame rates 2022-12-15 19:55:07 -08:00
EmosewaMC
e1cc25759e Merge remote-tracking branch 'upstream/main' into ServerShutdown 2022-12-15 18:43:28 -08:00
David Markowitz
b7341c8106
Resolve warnings, change config init order and remove unused Game variables for all servers (#877)
* Resolve warnings and change init order

Initialize dConfig first, before logger so we know whether or not to log to console
Initialize namespace Game variables to nullptr so they are a known value if accessed before initialization.
Removed unused Game variables
Replaced config with a pointer instead of referencing something on the stack.
Assign return values to system calls to silence warnings.

Tested that the server still compiles, runs and allows me to load into the game.

* Only start Master of config files exist

Also default the logging to console to on on the off chance the files exist but are wrong / corrupted.
2022-12-15 08:13:49 -06:00
EmosewaMC
1afe717563 Properly exit
Properly exit based on the path taken to shutdown master.

Tested that shutting down through sigint or sigterm returns -1
Tested that a segfault exits the program properly
Need to test that players who are trying to connect while master is shutting down are not able to spawn more child worlds.
2022-12-15 05:46:03 -08:00
EmosewaMC
435761f64b Improve shutdown 2022-12-15 04:14:00 -08:00
EmosewaMC
a14e16237b Only start Master of config files exist
Also default the logging to console to on on the off chance the files exist but are wrong / corrupted.
2022-12-06 19:30:43 -08:00
EmosewaMC
46f085eb4b Resolve warnings and change init order
Initialize dConfig first, before logger so we know whether or not to log to console
Initialize namespace Game variables to nullptr so they are a known value if accessed before initialization.
Removed unused Game variables
Replaced config with a pointer instead of referencing something on the stack.
Assign return values to system calls to silence warnings.

Tested that the server still compiles, runs and allows me to load into the game.
2022-12-06 04:39:09 -08:00
David Markowitz
18a0ae599b
Add bandwidth limit of 10kb/s(#863) 2022-12-05 16:08:47 -08:00
David Markowitz
2ba3103a0c
Implement FDB to SQLite (#872) 2022-12-05 00:57:58 -08:00
David Markowitz
e8ba3357e8
Add support to reload the config (#868) 2022-12-04 16:25:58 -06:00
Jonathan Romano
f8f5b731f1
Allow servers to be run from directories other than build. Read/write files relative to binary instead of cwd (#834)
Allows the server to be run from a non-build directory.  Also only read or write files relative to the build directory, regardless of where the server is run from
2022-11-27 03:59:59 -08:00
Jett
b974eed8f5
Make changes to certain database functions and a debug assert (#804)
- Replace all interaction of std::string and sqlString.
- Add a return before a debug assertion can be triggered by lvl chunks being loaded on server start.
2022-11-02 22:53:45 -05:00
Aaron Kimbrell
353c328485
compile fixes and default client_location (#809)
* support for gcc9 on ubuntu 18.04
This is needed to make filesystem work

* fix default for client location
2022-11-02 22:05:52 -05:00
Jett
4a6f3e44ee
Add support for packed clients (#802)
* First iteration of pack reader and interface

* Fix memory leak and remove logs

* Complete packed asset interface and begin on file loading replacement

* Implement proper BinaryIO error

* Improve AssetMemoryBuffer for reading and implement more reading

* Repair more file loading code and improve how navmeshes are loaded

* Missing checks implementation

* Revert addition of Manifest class and migration changes

* Resolved all feedback.
2022-11-01 13:21:26 -05:00
David Markowitz
906887bda9
Add automatic cdclient migration runner support and setup (#789)
* Add automatic migrations for CDServer

Add support to automatically migrate and update CDServers with new migrations.  Also adds support to simplify the setup process by simply putting the fdb in the res folder and letting the server convert it to sqlite.

This reduces the amount of back and forth when setting up a server.

* Remove transaction language

* Add DML execution
`poggers`
Add a way to execute DML commands through the sqlite connection on the server.

* Make DML Commands more robust

On the off chance the server is shutdown before the whole migration is run, lets just not add it to our "finished list" until the whole file is done.

* Update README
2022-10-30 00:38:43 -07:00
David Markowitz
c13937bd1f
Address Brick-By-Brick builds not properly saving and make migrations automatic (#725)
* Properly store BBB in database

Store the BBB data in the database as the received SD0 packet as opposed to just the raw lxfml.  Addressed several memory leaks as well.

* Add Sd0Conversion

Add brick by brick conversion commands with 2 parameters to tell the program what to do with the data.

Add zlib -> sd0 conversion.  Files look good at a glance but should be tested in game to ensure stability.  Tests to come.

* moving to laptop

ignore this commit.  I need to move this to my laptop

* Add functionality to delete bad models

Adds functionality to delete bad models.  Models are batched together and deleted in one commit.

More testing is needed to ensure data safety.  Positive tests on a live database reveal the broken models were truncated and complete ones were kept around successfully.  Tests should be done to ensure larger sd0 models are properly saved and not truncated since this command should be able to be run any time.

Valgrind tests need to be run as well to ensure no memory leaks exist.

* Delete from query change

Changed from delete to delete cascade and instead deleting from properties_contents as opposed to ugc.

* Address numerous bugs

DELETE CASCADE is not a valid SQL command so this was changed to a better delete statement.

Added user confirmation before deleting a broken model.
Address appending the string model appending bad data, causing excess deletion.
Addressed memory leaks with sql::Blob

* Error handling for string

* Even more proper handling...

* Add bounds check for cli command

Output a message if a bad command is used.

Update MasterServer.cpp

* Remove user interference

-Add back in mariadb build jobs so i dont nuke others systems
- Remove all user interference and consolidate work into one command since 1 depends on the next.

* Add comments

test

Revert "test"

This reverts commit fb831f268b7a2f0ccd20595aff64902ab4f4b4ee.

* Update CMakeMariaDBLists.txt

Test

* Improve migration runner

Migration runner now runs automatically.
- Resolved an issue where extremely large sql queries caused the database to go into an invalid state.
- Made migrations run automatically on server start.
- Resolved a tiny memory leak in migration runner? (discarded returned pointer)
- Moved sd0 migrations of brick models to be run automatically with migration runner.
- Created dummy file to tell when brick migrations have been run.

* Update README

Updated the README to reflect the new server migration state.

* Make model deleter actually delete models

My complicated sql actually did nothing...  Tested that this new SQL properly gets rid of bad data.

* Revert "Update CMakeMariaDBLists.txt"

This reverts commit 8b859d8529.
2022-10-24 17:20:36 -05:00
aronwk-aaron
19e77a38d8 format codebase 2022-07-28 08:39:57 -05:00
Aaron Kimbrell
e97ae92624
Make logger automatically put a newline (#675)
at the end of the line
remove all the newlines in log calls
2022-07-24 21:26:51 -05:00
Demetri Van Sickle
ec4ed8fa7e
Adding migration command to build script (updated) (#653)
* Added checks for migration runner

* Added migration command to build script
2022-07-18 18:44:21 -05:00
David Markowitz
b55606d41e
Fix the issue where we would create new worlds on ports that were still being used (#625)
* Wait for world to shutdown

We need to wait for the message that the world has shutdown before shutting it down.

* Dont sent people to dead instances

* Added shutting down check

Added check for isShuttingDown

* Update when we remove from master
2022-07-18 18:08:33 -05:00
Jett
77d35019cc
Replace the usage of RakString (#648) 2022-07-17 04:40:46 +01:00
Jett
d642de9462 Implement a migration runner 2022-07-12 02:16:08 +01:00
David Markowitz
325dc5a571
Revert "Wait for world to shutdown (#624)" (#626)
This reverts commit 66edf57b02.
2022-07-09 15:57:45 -07:00
David Markowitz
66edf57b02
Wait for world to shutdown (#624)
We need to wait for the message that the world has shutdown before shutting it down.
2022-07-09 15:27:20 -07:00
jumpy-badger
eaa962f265
Add Aarch64 support (#231)
* added mariadb-connector-cpp submodule

* raknet aarch64 support

* fix compile errors

* mariadb connector swap (in progress)

* update CMakeLists, add preprocessor definition to switch between mysql and mariadb connectors

* update types with missing aarch64 check

* corrected adding extra flag to properly compile mariadbconn in CMakeLists

* updated readme with arm builds section

* fix build failure if test folder does not exist

* Remove mysql connector from all builds, add mariadbconnector to windows build

* readd Linux check for backtrace lib to CMakeLists.txt

* Separate system specific mariadbconncpp extra compile flags

* Copy dlls to exes directory once built

* fetch prebuilt binaries on windows so that ClangCL can be used

* Delay load dll so that plugin directory is set correctly

* Fixed typo in glibcxx compile flag

* whitespacing, spaces -> tabs

* Updated README.md, included instructions to update

* Updated README.md

added libssl-dev requirement and removed mysql connector references from macOS builds section

* apple compile fixes for zlib and shared library name

* add windows arm64 checks to raknet

* remove extra . in shared library location

* Setup plugins directory for the connector to search in, pass openssl_root_dir on for apple

* Fix copy paths for single config generators and non windows

* change plugin folder location, another single config generator fix

* GENERATOR_IS_MULTI_CONFIG is a property not a variable

* Fixed a few errors after merge

* Fix plugin directory path, force windows to look at the right folder

* fixed directory name for make_directory command

* Update README.md

Updated MacOS, Windows build instructions.

* set INSTALL_PLUGINDIR so that the right directory is used

* Support for relative rpath for docker build

* added mariadb-connector-cpp submodule

* raknet aarch64 support

* fix compile errors

* mariadb connector swap (in progress)

* update CMakeLists, add preprocessor definition to switch between mysql and mariadb connectors

* update types with missing aarch64 check

* corrected adding extra flag to properly compile mariadbconn in CMakeLists

* updated readme with arm builds section

* fix build failure if test folder does not exist

* Remove mysql connector from all builds, add mariadbconnector to windows build

* readd Linux check for backtrace lib to CMakeLists.txt

* Separate system specific mariadbconncpp extra compile flags

* Copy dlls to exes directory once built

* fetch prebuilt binaries on windows so that ClangCL can be used

* Delay load dll so that plugin directory is set correctly

* Fixed typo in glibcxx compile flag

* whitespacing, spaces -> tabs

* Updated README.md, included instructions to update

* Updated README.md

added libssl-dev requirement and removed mysql connector references from macOS builds section

* apple compile fixes for zlib and shared library name

* add windows arm64 checks to raknet

* Setup plugins directory for the connector to search in, pass openssl_root_dir on for apple

* Fix copy paths for single config generators and non windows

* change plugin folder location, another single config generator fix

* GENERATOR_IS_MULTI_CONFIG is a property not a variable

* Fixed a few errors after merge

* Fix plugin directory path, force windows to look at the right folder

* fixed directory name for make_directory command

* Update README.md

Updated MacOS, Windows build instructions.

* set INSTALL_PLUGINDIR so that the right directory is used

* Support for relative rpath for docker build

* Rebase on main

* Remove extra git submodule

* Update CMakeLists.txt

* Remove CMakeLists.txt file from mariadb

Remove the CMakeLists.txt file from the mariaDBConnector so we dont build the tests.  Also add a config option to the CMakeVariables.txt so you can build the connector with multiple jobs

* Compile on windows

Specify the mariadbcpp.dll file location with a defined absolute path so windows knows it actually exists.

* default to 1 job

Default mariadb jobs running in parallel to 1 instead of 4

* Move mariadbcpp.dll file to the expected directory on windows

* Changed plugin

Updated the plugin location from the project binary directory to the expected location, the mariadb binary directory.

* Addressed windows dll issues by moving files to the expected directory instead of a directory that wouldnt get created

* Update README

Co-authored-by: Aaron Kimbrell <aronwk.aaron@gmail.com>
Co-authored-by: EmosewaMC <39972741+EmosewaMC@users.noreply.github.com>
2022-07-03 21:33:05 -07:00
EmosewaMC
6927bc614e Spacing 2022-04-09 18:33:38 -07:00
EmosewaMC
aee3290248 Update MasterServer.cpp 2022-04-09 16:02:12 -07:00