refactor: Database abstraction and organization of files (#1274)

* Database: Convert to proper namespace

* Database: Use base class and getter

* Database: Move files around

* Database: Add property Management query

Database: Move over user queries

Tested at gm 0 that pre-approved names are pre-approved, unapproved need moderator approval
deleting characters deletes the selcted one
refreshing the character page shows the last character you logged in as
tested all my characters show up when i login
tested that you can delete all 4 characters and the correct character is selected each time
tested renaming, approving names as gm0

Database: Add ugc model getter

Hey it works, look I got around the mariadb issue.

Database: Add queries

Database: consolidate name query

Database: Add friends list query

Update name of approved names query

Documentation

Database: Add name check

Database: Add BFF Query

Database: Move BFF Setter

Database: Move new friend query

Database: Add remove friend queries

Database: Add activity log

Database: Add ugc & prop content removal

Database: Add model update

Database: Add migration queries

Database: Add character and xml queries

Database: Add user queries

Untested, but compiling code

Need to test that new character names are properly assigned in the following scenarios
gm 0 and pre-approved name
gm 0 and unapproved name
gm 9 and pre-approved name
gm 9 and unapproved name

Database: constify function arguments

Database: Add pet queries

* Database: Move property model queries

Untested.  Need to test
placing a new model
moving existing one
removing ugc model
placing ugc model
moving ugc model(?)
changing privacy option variously
change description and name
approve property
can properly travel to property

* Property: Move stale reference deletion

* Database: Move performance update query

* Database: Add bug report query

* Database: Add cheat detection query

* Database: Add mail send query

* Untested code

need to test mailing from slash command, from all users of SendMail, getting bbb of a property and sending messages to bffs

* Update CDComponentsRegistryTable.h

Database: Rename and add further comments

Datavbase: Add comments

Add some comments

Build: Fix PCH directories

Database: Fix time

thanks apple

Database: Fix compiler warnings

Overload destructor
Define specialty for time_t
Use string instead of string_view for temp empty string

Update CDTable.h

Property: Update queries to use mapId

Database: Reorganize

Reorganize into CDClient folder and GameDatabase folder for clearer meanings and file structure

Folders: Rename to GameDatabase

MySQL: Remove MySQL Specifier from table

Database: Move Tables to Interfaces

Database: Reorder functions in header

Database: Simplify property queries

Database: Remove unused queries

Remove extra query definitions as well

Database: Consolidate User getters

Database: Comment logs

Update MySQLDatabase.cpp

Database: Use generic code

Playkey: Fix bad optional access

Database: Move stuff around

WorldServer: Update queries

Ugc reduced by many scopes
use new queries
very fast
tested that ugc still loads

Database: Add auth queries

I tested that only the correct password can sign into an account.
Tested that disabled playkeys do not allow the user to play the game

Database: Add donation query

Database: add objectId queries

Database: Add master queries

Database: Fix mis-named function

Database: Add slash command queries

Mail: Fix itemId type

CharFilter: Use new query

ObjectID: Remove duplicate code

SlashCommand: Update query with function

Database: Add mail queries

Ugc: Fix issues with saving models

Resolve large scope blocks as well

* Database: Add debug try catch rethrow macro

* General fixes

* fix play key not working

* Further fixes

---------

Co-authored-by: Aaron Kimbre <aronwk.aaron@gmail.com>
This commit is contained in:
David Markowitz
2023-11-17 16:47:18 -08:00
committed by GitHub
parent b68823b4cb
commit 7f623d358c
161 changed files with 2114 additions and 1516 deletions

View File

@@ -3,6 +3,8 @@
// Custom Classes
#include "CDTable.h"
#include <unordered_map>
enum class eReplicaComponentType : uint32_t;
struct CDComponentsRegistry {
unsigned int id; //!< The LOT is used as the ID

View File

@@ -1,4 +1,4 @@
set(DDATABASE_TABLES_SOURCES "CDActivitiesTable.cpp"
set(DDATABASE_CDCLIENTDATABASE_CDCLIENTTABLES_SOURCES "CDActivitiesTable.cpp"
"CDActivityRewardsTable.cpp"
"CDAnimationsTable.cpp"
"CDBehaviorParameterTable.cpp"

View File

@@ -0,0 +1,12 @@
set(DDATABASE_CDCLIENTDATABASE_SOURCES
"CDClientDatabase.cpp"
"CDClientManager.cpp"
)
add_subdirectory(CDClientTables)
foreach(file ${DDATABASE_CDCLIENTDATABASE_CDCLIENTTABLES_SOURCES})
set(DDATABASE_CDCLIENTDATABASE_SOURCES ${DDATABASE_CDCLIENTDATABASE_SOURCES} "CDClientTables/${file}")
endforeach()
set(DDATABASE_CDCLIENTDATABASE_SOURCES ${DDATABASE_CDCLIENTDATABASE_SOURCES} PARENT_SCOPE)

View File

@@ -1,12 +1,15 @@
set(DDATABASE_SOURCES "CDClientDatabase.cpp"
"CDClientManager.cpp"
"Database.cpp"
"MigrationRunner.cpp")
set(DDATABASE_SOURCES)
add_subdirectory(Tables)
add_subdirectory(CDClientDatabase)
foreach(file ${DDATABASE_TABLES_SOURCES})
set(DDATABASE_SOURCES ${DDATABASE_SOURCES} "Tables/${file}")
foreach(file ${DDATABASE_CDCLIENTDATABASE_SOURCES})
set(DDATABASE_SOURCES ${DDATABASE_SOURCES} "CDClientDatabase/${file}")
endforeach()
add_subdirectory(GameDatabase)
foreach(file ${DDATABASE_GAMEDATABASE_SOURCES})
set(DDATABASE_SOURCES ${DDATABASE_SOURCES} "GameDatabase/${file}")
endforeach()
add_library(dDatabase STATIC ${DDATABASE_SOURCES})

View File

@@ -1,118 +0,0 @@
#include "Database.h"
#include "Game.h"
#include "dConfig.h"
#include "Logger.h"
using namespace std;
#pragma warning (disable:4251) //Disables SQL warnings
sql::Driver* Database::driver;
sql::Connection* Database::con;
sql::Properties Database::props;
std::string Database::database;
void Database::Connect(const string& host, const string& database, const string& username, const string& password) {
//To bypass debug issues:
const char* szDatabase = database.c_str();
const char* szUsername = username.c_str();
const char* szPassword = password.c_str();
driver = sql::mariadb::get_driver_instance();
sql::Properties properties;
// The mariadb connector is *supposed* to handle unix:// and pipe:// prefixes to hostName, but there are bugs where
// 1) it tries to parse a database from the connection string (like in tcp://localhost:3001/darkflame) based on the
// presence of a /
// 2) even avoiding that, the connector still assumes you're connecting with a tcp socket
// So, what we do in the presence of a unix socket or pipe is to set the hostname to the protocol and localhost,
// which avoids parsing errors while still ensuring the correct connection type is used, and then setting the appropriate
// property manually (which the URL parsing fails to do)
const std::string UNIX_PROTO = "unix://";
const std::string PIPE_PROTO = "pipe://";
if (host.find(UNIX_PROTO) == 0) {
properties["hostName"] = "unix://localhost";
properties["localSocket"] = host.substr(UNIX_PROTO.length()).c_str();
} else if (host.find(PIPE_PROTO) == 0) {
properties["hostName"] = "pipe://localhost";
properties["pipe"] = host.substr(PIPE_PROTO.length()).c_str();
} else {
properties["hostName"] = host.c_str();
}
properties["user"] = szUsername;
properties["password"] = szPassword;
properties["autoReconnect"] = "true";
Database::props = properties;
Database::database = database;
Database::Connect();
}
void Database::Connect() {
// `connect(const Properties& props)` segfaults in windows debug, but
// `connect(const SQLString& host, const SQLString& user, const SQLString& pwd)` doesn't handle pipes/unix sockets correctly
if (Database::props.find("localSocket") != Database::props.end() || Database::props.find("pipe") != Database::props.end()) {
con = driver->connect(Database::props);
} else {
con = driver->connect(Database::props["hostName"].c_str(), Database::props["user"].c_str(), Database::props["password"].c_str());
}
con->setSchema(Database::database.c_str());
}
void Database::Destroy(std::string source, bool log) {
if (!con) return;
if (log) {
if (source != "") LOG("Destroying MySQL connection from %s!", source.c_str());
else LOG("Destroying MySQL connection!");
}
con->close();
delete con;
} //Destroy
sql::Statement* Database::CreateStmt() {
sql::Statement* toReturn = con->createStatement();
return toReturn;
} //CreateStmt
sql::PreparedStatement* Database::CreatePreppedStmt(const std::string& query) {
const char* test = query.c_str();
size_t size = query.length();
sql::SQLString str(test, size);
if (!con) {
Connect();
LOG("Trying to reconnect to MySQL");
}
if (!con->isValid() || con->isClosed()) {
delete con;
con = nullptr;
Connect();
LOG("Trying to reconnect to MySQL from invalid or closed connection");
}
auto* stmt = con->prepareStatement(str);
return stmt;
} //CreatePreppedStmt
void Database::Commit() {
Database::con->commit();
}
bool Database::GetAutoCommit() {
// TODO This should not just access a pointer. A future PR should update this
// to check for null and throw an error if the connection is not valid.
return con->getAutoCommit();
}
void Database::SetAutoCommit(bool value) {
// TODO This should not just access a pointer. A future PR should update this
// to check for null and throw an error if the connection is not valid.
Database::con->setAutoCommit(value);
}

View File

@@ -1,31 +0,0 @@
#pragma once
#include <string>
#include <conncpp.hpp>
class MySqlException : public std::runtime_error {
public:
MySqlException() : std::runtime_error("MySQL error!") {}
MySqlException(const std::string& msg) : std::runtime_error(msg.c_str()) {}
};
class Database {
private:
static sql::Driver* driver;
static sql::Connection* con;
static sql::Properties props;
static std::string database;
public:
static void Connect(const std::string& host, const std::string& database, const std::string& username, const std::string& password);
static void Connect();
static void Destroy(std::string source = "", bool log = true);
static sql::Statement* CreateStmt();
static sql::PreparedStatement* CreatePreppedStmt(const std::string& query);
static void Commit();
static bool GetAutoCommit();
static void SetAutoCommit(bool value);
static std::string GetDatabase() { return database; }
static sql::Properties GetProperties() { return props; }
};

View File

@@ -0,0 +1,12 @@
set(DDATABASE_GAMEDATABASE_SOURCES
"Database.cpp"
"MigrationRunner.cpp"
)
add_subdirectory(MySQL)
foreach(file ${DDATABSE_DATABSES_MYSQL_SOURCES})
set(DDATABASE_GAMEDATABASE_SOURCES ${DDATABASE_GAMEDATABASE_SOURCES} "MySQL/${file}")
endforeach()
set(DDATABASE_GAMEDATABASE_SOURCES ${DDATABASE_GAMEDATABASE_SOURCES} PARENT_SCOPE)

View File

@@ -0,0 +1,40 @@
#include "Database.h"
#include "Game.h"
#include "dConfig.h"
#include "Logger.h"
#include "MySQLDatabase.h"
#include "DluAssert.h"
#pragma warning (disable:4251) //Disables SQL warnings
namespace {
GameDatabase* database = nullptr;
}
void Database::Connect() {
if (database) {
LOG("Tried to connect to database when it's already connected!");
return;
}
database = new MySQLDatabase();
database->Connect();
}
GameDatabase* Database::Get() {
if (!database) {
LOG("Tried to get database when it's not connected!");
Connect();
}
return database;
}
void Database::Destroy(std::string source) {
if (database) {
database->Destroy(source);
delete database;
database = nullptr;
} else {
LOG("Trying to destroy database when it's not connected!");
}
}

View File

@@ -0,0 +1,12 @@
#pragma once
#include <string>
#include <conncpp.hpp>
#include "GameDatabase.h"
namespace Database {
void Connect();
GameDatabase* Get();
void Destroy(std::string source = "");
};

View File

@@ -0,0 +1,55 @@
#ifndef __GAMEDATABASE__H__
#define __GAMEDATABASE__H__
#include <optional>
#include "ILeaderboard.h"
#include "IPlayerCheatDetections.h"
#include "ICommandLog.h"
#include "IMail.h"
#include "IObjectIdTracker.h"
#include "IPlayKeys.h"
#include "IServers.h"
#include "IBugReports.h"
#include "IPropertyContents.h"
#include "IProperty.h"
#include "IPetNames.h"
#include "ICharXml.h"
#include "IMigrationHistory.h"
#include "IUgc.h"
#include "IFriends.h"
#include "ICharInfo.h"
#include "IAccounts.h"
#include "IActivityLog.h"
namespace sql {
class Statement;
class PreparedStatement;
};
#ifdef _DEBUG
# define DLU_SQL_TRY_CATCH_RETHROW(x) do { try { x; } catch (sql::SQLException& ex) { LOG("SQL Error: %s", ex.what()); throw; } } while(0)
#else
# define DLU_SQL_TRY_CATCH_RETHROW(x) x
#endif // _DEBUG
class GameDatabase :
public IPlayKeys, public ILeaderboard, public IObjectIdTracker, public IServers,
public IMail, public ICommandLog, public IPlayerCheatDetections, public IBugReports,
public IPropertyContents, public IProperty, public IPetNames, public ICharXml,
public IMigrationHistory, public IUgc, public IFriends, public ICharInfo,
public IAccounts, public IActivityLog {
public:
virtual ~GameDatabase() = default;
// TODO: These should be made private.
virtual void Connect() = 0;
virtual void Destroy(std::string source = "") = 0;
virtual void ExecuteCustomQuery(const std::string_view query) = 0;
virtual sql::PreparedStatement* CreatePreppedStmt(const std::string& query) = 0;
virtual void Commit() = 0;
virtual bool GetAutoCommit() = 0;
virtual void SetAutoCommit(bool value) = 0;
virtual void DeleteCharacter(const uint32_t characterId) = 0;
};
#endif //!__GAMEDATABASE__H__

View File

@@ -0,0 +1,37 @@
#ifndef __IACCOUNTS__H__
#define __IACCOUNTS__H__
#include <cstdint>
#include <optional>
#include <string_view>
enum class eGameMasterLevel : uint8_t;
class IAccounts {
public:
struct Info {
std::string bcryptPassword;
uint32_t id{};
uint32_t playKeyId{};
bool banned{};
bool locked{};
eGameMasterLevel maxGmLevel{};
};
// Get the account info for the given username.
virtual std::optional<IAccounts::Info> GetAccountInfo(const std::string_view username) = 0;
// Update the account's unmute time.
virtual void UpdateAccountUnmuteTime(const uint32_t accountId, const uint64_t timeToUnmute) = 0;
// Update the account's ban status.
virtual void UpdateAccountBan(const uint32_t accountId, const bool banned) = 0;
// Update the account's password.
virtual void UpdateAccountPassword(const uint32_t accountId, const std::string_view bcryptpassword) = 0;
// Add a new account to the database.
virtual void InsertNewAccount(const std::string_view username, const std::string_view bcryptpassword) = 0;
};
#endif //!__IACCOUNTS__H__

View File

@@ -0,0 +1,19 @@
#ifndef __IACTIVITYLOG__H__
#define __IACTIVITYLOG__H__
#include <cstdint>
#include "dCommonVars.h"
enum class eActivityType : uint32_t {
PlayerLoggedIn,
PlayerLoggedOut,
};
class IActivityLog {
public:
// Update the activity log for the given account.
virtual void UpdateActivityLog(const uint32_t characterId, const eActivityType activityType, const LWOMAPID mapId) = 0;
};
#endif //!__IACTIVITYLOG__H__

View File

@@ -0,0 +1,20 @@
#ifndef __IBUGREPORTS__H__
#define __IBUGREPORTS__H__
#include <cstdint>
#include <string_view>
class IBugReports {
public:
struct Info {
std::string body;
std::string clientVersion;
std::string otherPlayer;
std::string selection;
uint32_t characterId{};
};
// Add a new bug report to the database.
virtual void InsertNewBugReport(const Info& info) = 0;
};
#endif //!__IBUGREPORTS__H__

View File

@@ -0,0 +1,49 @@
#ifndef __ICHARINFO__H__
#define __ICHARINFO__H__
#include <cstdint>
#include <optional>
#include <string>
#include <string_view>
#include <vector>
#include "ePermissionMap.h"
class ICharInfo {
public:
struct Info {
std::string name;
std::string pendingName;
uint32_t id{};
uint32_t accountId{};
bool needsRename{};
LWOCLONEID cloneId{};
ePermissionMap permissionMap{};
};
// Get the approved names of all characters.
virtual std::vector<std::string> GetApprovedCharacterNames() = 0;
// Get the character info for the given character id.
virtual std::optional<ICharInfo::Info> GetCharacterInfo(const uint32_t charId) = 0;
// Get the character info for the given character name.
virtual std::optional<ICharInfo::Info> GetCharacterInfo(const std::string_view name) = 0;
// Get the character ids for the given account.
virtual std::vector<uint32_t> GetAccountCharacterIds(const uint32_t accountId) = 0;
// Insert a new character into the database.
virtual void InsertNewCharacter(const ICharInfo::Info info) = 0;
// Set the name of the given character.
virtual void SetCharacterName(const uint32_t characterId, const std::string_view name) = 0;
// Set the pending name of the given character.
virtual void SetPendingCharacterName(const uint32_t characterId, const std::string_view name) = 0;
// Updates the given character ids last login to be right now.
virtual void UpdateLastLoggedInCharacter(const uint32_t characterId) = 0;
};
#endif //!__ICHARINFO__H__

View File

@@ -0,0 +1,20 @@
#ifndef __ICHARXML__H__
#define __ICHARXML__H__
#include <cstdint>
#include <string>
#include <string_view>
class ICharXml {
public:
// Get the character xml for the given character id.
virtual std::string GetCharacterXml(const uint32_t charId) = 0;
// Update the character xml for the given character id.
virtual void UpdateCharacterXml(const uint32_t charId, const std::string_view lxfml) = 0;
// Insert the character xml for the given character id.
virtual void InsertCharacterXml(const uint32_t characterId, const std::string_view lxfml) = 0;
};
#endif //!__ICHARXML__H__

View File

@@ -0,0 +1,14 @@
#ifndef __ICOMMANDLOG__H__
#define __ICOMMANDLOG__H__
#include <cstdint>
#include <string_view>
class ICommandLog {
public:
// Insert a new slash command log entry.
virtual void InsertSlashCommandUsage(const uint32_t characterId, const std::string_view command) = 0;
};
#endif //!__ICOMMANDLOG__H__

View File

@@ -0,0 +1,32 @@
#ifndef __IFRIENDS__H__
#define __IFRIENDS__H__
#include <cstdint>
#include <optional>
#include <vector>
class IFriends {
public:
struct BestFriendStatus {
uint32_t playerCharacterId{};
uint32_t friendCharacterId{};
uint32_t bestFriendStatus{};
};
// Get the friends list for the given character id.
virtual std::vector<FriendData> GetFriendsList(const uint32_t charId) = 0;
// Get the best friend status for the given player and friend character ids.
virtual std::optional<IFriends::BestFriendStatus> GetBestFriendStatus(const uint32_t playerCharacterId, const uint32_t friendCharacterId) = 0;
// Set the best friend status for the given player and friend character ids.
virtual void SetBestFriendStatus(const uint32_t playerCharacterId, const uint32_t friendCharacterId, const uint32_t bestFriendStatus) = 0;
// Add a friend to the given character id.
virtual void AddFriend(const uint32_t playerCharacterId, const uint32_t friendCharacterId) = 0;
// Remove a friend from the given character id.
virtual void RemoveFriend(const uint32_t playerCharacterId, const uint32_t friendCharacterId) = 0;
};
#endif //!__IFRIENDS__H__

View File

@@ -0,0 +1,14 @@
#ifndef __ILEADERBOARD__H__
#define __ILEADERBOARD__H__
#include <cstdint>
#include <optional>
class ILeaderboard {
public:
// Get the donation total for the given activity id.
virtual std::optional<uint32_t> GetDonationTotal(const uint32_t activityId) = 0;
};
#endif //!__ILEADERBOARD__H__

View File

@@ -0,0 +1,54 @@
#ifndef __IMAIL__H__
#define __IMAIL__H__
#include <cstdint>
#include <optional>
#include <string_view>
#include "dCommonVars.h"
#include "NiQuaternion.h"
#include "NiPoint3.h"
class IMail {
public:
struct MailInfo {
std::string senderUsername;
std::string recipient;
std::string subject;
std::string body;
uint64_t id{};
uint32_t senderId{};
uint32_t receiverId{};
uint64_t timeSent{};
bool wasRead{};
struct {
LWOOBJID itemID{};
int32_t itemCount{};
LOT itemLOT{};
LWOOBJID itemSubkey{};
};
};
// Insert a new mail into the database.
virtual void InsertNewMail(const MailInfo& mail) = 0;
// Get the mail for the given character id.
virtual std::vector<MailInfo> GetMailForPlayer(const uint32_t characterId, const uint32_t numberOfMail) = 0;
// Get the mail for the given mail id.
virtual std::optional<MailInfo> GetMail(const uint64_t mailId) = 0;
// Get the number of unread mail for the given character id.
virtual uint32_t GetUnreadMailCount(const uint32_t characterId) = 0;
// Mark the given mail as read.
virtual void MarkMailRead(const uint64_t mailId) = 0;
// Claim the item from the given mail.
virtual void ClaimMailItem(const uint64_t mailId) = 0;
// Delete the given mail.
virtual void DeleteMail(const uint64_t mailId) = 0;
};
#endif //!__IMAIL__H__

View File

@@ -0,0 +1,17 @@
#ifndef __IMIGRATIONHISTORY__H__
#define __IMIGRATIONHISTORY__H__
#include <string_view>
class IMigrationHistory {
public:
// Create the migration history table.
virtual void CreateMigrationHistoryTable() = 0;
// Check if the given migration has been run.
virtual bool IsMigrationRun(const std::string_view str) = 0;
// Insert the given migration into the migration history table.
virtual void InsertMigration(const std::string_view str) = 0;
};
#endif //!__IMIGRATIONHISTORY__H__

Some files were not shown because too many files have changed in this diff Show More