refactor: Database abstraction and organization of files (#1274)

* Database: Convert to proper namespace

* Database: Use base class and getter

* Database: Move files around

* Database: Add property Management query

Database: Move over user queries

Tested at gm 0 that pre-approved names are pre-approved, unapproved need moderator approval
deleting characters deletes the selcted one
refreshing the character page shows the last character you logged in as
tested all my characters show up when i login
tested that you can delete all 4 characters and the correct character is selected each time
tested renaming, approving names as gm0

Database: Add ugc model getter

Hey it works, look I got around the mariadb issue.

Database: Add queries

Database: consolidate name query

Database: Add friends list query

Update name of approved names query

Documentation

Database: Add name check

Database: Add BFF Query

Database: Move BFF Setter

Database: Move new friend query

Database: Add remove friend queries

Database: Add activity log

Database: Add ugc & prop content removal

Database: Add model update

Database: Add migration queries

Database: Add character and xml queries

Database: Add user queries

Untested, but compiling code

Need to test that new character names are properly assigned in the following scenarios
gm 0 and pre-approved name
gm 0 and unapproved name
gm 9 and pre-approved name
gm 9 and unapproved name

Database: constify function arguments

Database: Add pet queries

* Database: Move property model queries

Untested.  Need to test
placing a new model
moving existing one
removing ugc model
placing ugc model
moving ugc model(?)
changing privacy option variously
change description and name
approve property
can properly travel to property

* Property: Move stale reference deletion

* Database: Move performance update query

* Database: Add bug report query

* Database: Add cheat detection query

* Database: Add mail send query

* Untested code

need to test mailing from slash command, from all users of SendMail, getting bbb of a property and sending messages to bffs

* Update CDComponentsRegistryTable.h

Database: Rename and add further comments

Datavbase: Add comments

Add some comments

Build: Fix PCH directories

Database: Fix time

thanks apple

Database: Fix compiler warnings

Overload destructor
Define specialty for time_t
Use string instead of string_view for temp empty string

Update CDTable.h

Property: Update queries to use mapId

Database: Reorganize

Reorganize into CDClient folder and GameDatabase folder for clearer meanings and file structure

Folders: Rename to GameDatabase

MySQL: Remove MySQL Specifier from table

Database: Move Tables to Interfaces

Database: Reorder functions in header

Database: Simplify property queries

Database: Remove unused queries

Remove extra query definitions as well

Database: Consolidate User getters

Database: Comment logs

Update MySQLDatabase.cpp

Database: Use generic code

Playkey: Fix bad optional access

Database: Move stuff around

WorldServer: Update queries

Ugc reduced by many scopes
use new queries
very fast
tested that ugc still loads

Database: Add auth queries

I tested that only the correct password can sign into an account.
Tested that disabled playkeys do not allow the user to play the game

Database: Add donation query

Database: add objectId queries

Database: Add master queries

Database: Fix mis-named function

Database: Add slash command queries

Mail: Fix itemId type

CharFilter: Use new query

ObjectID: Remove duplicate code

SlashCommand: Update query with function

Database: Add mail queries

Ugc: Fix issues with saving models

Resolve large scope blocks as well

* Database: Add debug try catch rethrow macro

* General fixes

* fix play key not working

* Further fixes

---------

Co-authored-by: Aaron Kimbre <aronwk.aaron@gmail.com>

CMakeLists.txt

@@ -214,7 +214,12 @@ set(INCLUDED_DIRECTORIES
 	"dNavigation/dTerrain"
 	"dZoneManager"
 	"dDatabase"
-	"dDatabase/Tables"
+	"dDatabase/CDClientDatabase"
+	"dDatabase/CDClientDatabase/CDClientTables"
+	"dDatabase/GameDatabase"
+	"dDatabase/GameDatabase/ITables"
+	"dDatabase/GameDatabase/MySQL"
+	"dDatabase/GameDatabase/MySQL/Tables"
 	"dNet"
 	"dScripts"
 	"dScripts/02_server"
@@ -329,8 +334,9 @@ add_subdirectory(thirdparty)
 file(
 	GLOB HEADERS_DDATABASE
 	LIST_DIRECTORIES false
-	${PROJECT_SOURCE_DIR}/dDatabase/*.h
-	${PROJECT_SOURCE_DIR}/dDatabase/Tables/*.h
+	${PROJECT_SOURCE_DIR}/dDatabase/CDClientDatabase/*.h
+	${PROJECT_SOURCE_DIR}/dDatabase/CDClientDatabase/CDClientTables/*.h
+	${PROJECT_SOURCE_DIR}/dDatabase/GameDatabase/ITables/*.h
 	${PROJECT_SOURCE_DIR}/thirdparty/SQLite/*.h
 )
 

dAuthServer/AuthServer.cpp

@@ -55,14 +55,8 @@ int main(int argc, char** argv) {
 	LOG("Version: %i.%i", PROJECT_VERSION_MAJOR, PROJECT_VERSION_MINOR);
 	LOG("Compiled on: %s", __TIMESTAMP__);
 
-	//Connect to the MySQL Database
-	std::string mysql_host = Game::config->GetValue("mysql_host");
-	std::string mysql_database = Game::config->GetValue("mysql_database");
-	std::string mysql_username = Game::config->GetValue("mysql_username");
-	std::string mysql_password = Game::config->GetValue("mysql_password");
-
 	try {
-		Database::Connect(mysql_host, mysql_database, mysql_username, mysql_password);
+		Database::Connect();
 	} catch (sql::SQLException& ex) {
 		LOG("Got an error while connecting to the database: %s", ex.what());
 		Database::Destroy("AuthServer");
@@ -74,15 +68,12 @@ int main(int argc, char** argv) {
 	//Find out the master's IP:
 	std::string masterIP;
 	uint32_t masterPort = 1500;
-	sql::PreparedStatement* stmt = Database::CreatePreppedStmt("SELECT ip, port FROM servers WHERE name='master';");
-	auto res = stmt->executeQuery();
-	while (res->next()) {
-		masterIP = res->getString(1).c_str();
-		masterPort = res->getInt(2);
-	}
 
-	delete res;
-	delete stmt;
+	auto masterInfo = Database::Get()->GetMasterInfo();
+	if (masterInfo) {
+		masterIP = masterInfo->ip;
+		masterPort = masterInfo->port;
+	}
 
 	Game::randomEngine = std::mt19937(time(0));
 
@@ -134,16 +125,12 @@ int main(int argc, char** argv) {
 			//Find out the master's IP for absolutely no reason:
 			std::string masterIP;
 			uint32_t masterPort;
-			sql::PreparedStatement* stmt = Database::CreatePreppedStmt("SELECT ip, port FROM servers WHERE name='master';");
-			auto res = stmt->executeQuery();
-			while (res->next()) {
-				masterIP = res->getString(1).c_str();
-				masterPort = res->getInt(2);
+			auto masterInfo = Database::Get()->GetMasterInfo();
+			if (masterInfo) {
+				masterIP = masterInfo->ip;
+				masterPort = masterInfo->port;
 			}
 
-			delete res;
-			delete stmt;
-
 			framesSinceLastSQLPing = 0;
 		} else framesSinceLastSQLPing++;
 

dChatFilter/dChatFilter.cpp

@@ -32,15 +32,11 @@ dChatFilter::dChatFilter(const std::string& filepath, bool dontGenerateDCF) {
 	}
 
 	//Read player names that are ok as well:
-	auto stmt = Database::CreatePreppedStmt("select name from charinfo;");
-	auto res = stmt->executeQuery();
-	while (res->next()) {
-		std::string line = res->getString(1).c_str();
-		std::transform(line.begin(), line.end(), line.begin(), ::tolower); //Transform to lowercase
-		m_ApprovedWords.push_back(CalculateHash(line));
+	auto approvedNames = Database::Get()->GetApprovedCharacterNames();
+	for (auto& name : approvedNames) {
+		std::transform(name.begin(), name.end(), name.begin(), ::tolower); //Transform to lowercase
+		m_ApprovedWords.push_back(CalculateHash(name));
 	}
-	delete res;
-	delete stmt;
 }
 
 dChatFilter::~dChatFilter() {

dChatServer/ChatPacketHandler.cpp

@@ -30,32 +30,17 @@ void ChatPacketHandler::HandleFriendlistRequest(Packet* packet) {
 	auto player = playerContainer.GetPlayerData(playerID);
 	if (!player) return;
 
-	//Get our friends list from the Db.  Using a derived table since the friend of a player can be in either column.
-	std::unique_ptr<sql::PreparedStatement> stmt(Database::CreatePreppedStmt(
-		"SELECT fr.requested_player, best_friend, ci.name FROM "
-		"(SELECT CASE "
-		"WHEN player_id = ? THEN friend_id "
-		"WHEN friend_id = ? THEN player_id "
-		"END AS requested_player, best_friend FROM friends) AS fr "
-		"JOIN charinfo AS ci ON ci.id = fr.requested_player "
-		"WHERE fr.requested_player IS NOT NULL AND fr.requested_player != ?;"));
-	stmt->setUInt(1, static_cast<uint32_t>(playerID));
-	stmt->setUInt(2, static_cast<uint32_t>(playerID));
-	stmt->setUInt(3, static_cast<uint32_t>(playerID));
-
-	std::vector<FriendData> friends;
-
-	std::unique_ptr<sql::ResultSet> res(stmt->executeQuery());
-	while (res->next()) {
+	auto friendsList = Database::Get()->GetFriendsList(playerID);
+	for (const auto& friendData : friendsList) {
 		FriendData fd;
 		fd.isFTP = false; // not a thing in DLU
-		fd.friendID = res->getUInt(1);
+		fd.friendID = friendData.friendID;
 		GeneralUtils::SetBit(fd.friendID, eObjectBits::PERSISTENT);
 		GeneralUtils::SetBit(fd.friendID, eObjectBits::CHARACTER);
 
-		fd.isBestFriend = res->getInt(2) == 3; //0 = friends, 1 = left_requested, 2 = right_requested, 3 = both_accepted - are now bffs
+		fd.isBestFriend = friendData.isBestFriend; //0 = friends, 1 = left_requested, 2 = right_requested, 3 = both_accepted - are now bffs
 		if (fd.isBestFriend) player->countOfBestFriends += 1;
-		fd.friendName = res->getString(3);
+		fd.friendName = friendData.friendName;
 
 		//Now check if they're online:
 		auto fr = playerContainer.GetPlayerData(fd.friendID);
@@ -71,7 +56,7 @@ void ChatPacketHandler::HandleFriendlistRequest(Packet* packet) {
 			fd.zoneID = LWOZONEID();
 		}
 
-		friends.push_back(fd);
+		player->friends.push_back(fd);
 	}
 
 	//Now, we need to send the friendlist to the server they came from:
@@ -83,14 +68,12 @@ void ChatPacketHandler::HandleFriendlistRequest(Packet* packet) {
 	BitStreamUtils::WriteHeader(bitStream, eConnectionType::CLIENT, eClientMessageType::GET_FRIENDS_LIST_RESPONSE);
 	bitStream.Write<uint8_t>(0);
 	bitStream.Write<uint16_t>(1); //Length of packet -- just writing one as it doesn't matter, client skips it.
-	bitStream.Write((uint16_t)friends.size());
+	bitStream.Write((uint16_t)player->friends.size());
 
-	for (auto& data : friends) {
+	for (auto& data : player->friends) {
 		data.Serialize(bitStream);
 	}
 
-	player->friends = friends;
-
 	SystemAddress sysAddr = player->sysAddr;
 	SEND_PACKET;
 }
@@ -152,35 +135,26 @@ void ChatPacketHandler::HandleFriendRequest(Packet* packet) {
 	// If at this point we dont have a target, then they arent online and we cant send the request.
 	// Send the response code that corresponds to what the error is.
 	if (!requestee) {
-		std::unique_ptr<sql::PreparedStatement> nameQuery(Database::CreatePreppedStmt("SELECT name from charinfo where name = ?;"));
-		nameQuery->setString(1, playerName);
-		std::unique_ptr<sql::ResultSet> result(nameQuery->executeQuery());
-
 		requestee.reset(new PlayerData());
 		requestee->playerName = playerName;
+		auto responseType = Database::Get()->GetCharacterInfo(playerName)
+			? eAddFriendResponseType::NOTONLINE
+			: eAddFriendResponseType::INVALIDCHARACTER;
 
-		SendFriendResponse(requestor, requestee.get(), result->next() ? eAddFriendResponseType::NOTONLINE : eAddFriendResponseType::INVALIDCHARACTER);
+		SendFriendResponse(requestor, requestee.get(), responseType);
 		return;
 	}
 
 	if (isBestFriendRequest) {
-		std::unique_ptr<sql::PreparedStatement> friendUpdate(Database::CreatePreppedStmt("SELECT * FROM friends WHERE (player_id = ? AND friend_id = ?) OR (player_id = ? AND friend_id = ?) LIMIT 1;"));
-		friendUpdate->setUInt(1, static_cast<uint32_t>(requestorPlayerID));
-		friendUpdate->setUInt(2, static_cast<uint32_t>(requestee->playerID));
-		friendUpdate->setUInt(3, static_cast<uint32_t>(requestee->playerID));
-		friendUpdate->setUInt(4, static_cast<uint32_t>(requestorPlayerID));
-		std::unique_ptr<sql::ResultSet> result(friendUpdate->executeQuery());
 
-		LWOOBJID queryPlayerID = LWOOBJID_EMPTY;
-		LWOOBJID queryFriendID = LWOOBJID_EMPTY;
 		uint8_t oldBestFriendStatus{};
 		uint8_t bestFriendStatus{};
-
-		if (result->next()) {
+		auto bestFriendInfo = Database::Get()->GetBestFriendStatus(requestorPlayerID, requestee->playerID);
+		if (bestFriendInfo) {
 			// Get the IDs
-			queryPlayerID = result->getInt(1);
-			queryFriendID = result->getInt(2);
-			oldBestFriendStatus = result->getInt(3);
+			LWOOBJID queryPlayerID = bestFriendInfo->playerCharacterId;
+			LWOOBJID queryFriendID = bestFriendInfo->friendCharacterId;
+			oldBestFriendStatus = bestFriendInfo->bestFriendStatus;
 			bestFriendStatus = oldBestFriendStatus;
 
 			// Set the bits
@@ -211,13 +185,7 @@ void ChatPacketHandler::HandleFriendRequest(Packet* packet) {
 				}
 			} else {
 				// Then update the database with this new info.
-				std::unique_ptr<sql::PreparedStatement> updateQuery(Database::CreatePreppedStmt("UPDATE friends SET best_friend = ? WHERE (player_id = ? AND friend_id = ?) OR (player_id = ? AND friend_id = ?) LIMIT 1;"));
-				updateQuery->setUInt(1, bestFriendStatus);
-				updateQuery->setUInt(2, static_cast<uint32_t>(requestorPlayerID));
-				updateQuery->setUInt(3, static_cast<uint32_t>(requestee->playerID));
-				updateQuery->setUInt(4, static_cast<uint32_t>(requestee->playerID));
-				updateQuery->setUInt(5, static_cast<uint32_t>(requestorPlayerID));
-				updateQuery->executeUpdate();
+				Database::Get()->SetBestFriendStatus(requestorPlayerID, requestee->playerID, bestFriendStatus);
 				// Sent the best friend update here if the value is 3
 				if (bestFriendStatus == 3U) {
 					requestee->countOfBestFriends += 1;
@@ -319,11 +287,7 @@ void ChatPacketHandler::HandleFriendResponse(Packet* packet) {
 		requesteeData.isOnline = true;
 		requestor->friends.push_back(requesteeData);
 
-		std::unique_ptr<sql::PreparedStatement> statement(Database::CreatePreppedStmt("INSERT IGNORE INTO `friends` (`player_id`, `friend_id`, `best_friend`) VALUES (?,?,?);"));
-		statement->setUInt(1, static_cast<uint32_t>(requestor->playerID));
-		statement->setUInt(2, static_cast<uint32_t>(requestee->playerID));
-		statement->setInt(3, 0);
-		statement->execute();
+		Database::Get()->AddFriend(requestor->playerID, requestee->playerID);
 	}
 
 	if (serverResponseCode != eAddFriendResponseType::DECLINED) SendFriendResponse(requestor, requestee, serverResponseCode, isAlreadyBestFriends);
@@ -338,25 +302,17 @@ void ChatPacketHandler::HandleRemoveFriend(Packet* packet) {
 
 	//we'll have to query the db here to find the user, since you can delete them while they're offline.
 	//First, we need to find their ID:
-	std::unique_ptr<sql::PreparedStatement> stmt(Database::CreatePreppedStmt("SELECT id FROM charinfo WHERE name=? LIMIT 1;"));
-	stmt->setString(1, friendName.c_str());
-
 	LWOOBJID friendID = 0;
-	std::unique_ptr<sql::ResultSet> res(stmt->executeQuery());
-	while (res->next()) {
-		friendID = res->getUInt(1);
+	auto friendIdResult = Database::Get()->GetCharacterInfo(friendName);
+	if (friendIdResult) {
+		friendID = friendIdResult->id;
 	}
 
 	// Convert friendID to LWOOBJID
 	GeneralUtils::SetBit(friendID, eObjectBits::PERSISTENT);
 	GeneralUtils::SetBit(friendID, eObjectBits::CHARACTER);
 
-	std::unique_ptr<sql::PreparedStatement> deletestmt(Database::CreatePreppedStmt("DELETE FROM friends WHERE (player_id = ? AND friend_id = ?) OR (player_id = ? AND friend_id = ?) LIMIT 1;"));
-	deletestmt->setUInt(1, static_cast<uint32_t>(playerID));
-	deletestmt->setUInt(2, static_cast<uint32_t>(friendID));
-	deletestmt->setUInt(3, static_cast<uint32_t>(friendID));
-	deletestmt->setUInt(4, static_cast<uint32_t>(playerID));
-	deletestmt->execute();
+	Database::Get()->RemoveFriend(playerID, friendID);
 
 	//Now, we need to send an update to notify the sender (and possibly, receiver) that their friendship has been ended:
 	auto goonA = playerContainer.GetPlayerData(playerID);

dChatServer/ChatServer.cpp

@@ -78,13 +78,8 @@ int main(int argc, char** argv) {
 	}
 
 	//Connect to the MySQL Database
-	std::string mysql_host = Game::config->GetValue("mysql_host");
-	std::string mysql_database = Game::config->GetValue("mysql_database");
-	std::string mysql_username = Game::config->GetValue("mysql_username");
-	std::string mysql_password = Game::config->GetValue("mysql_password");
-
 	try {
-		Database::Connect(mysql_host, mysql_database, mysql_username, mysql_password);
+		Database::Connect();
 	} catch (sql::SQLException& ex) {
 		LOG("Got an error while connecting to the database: %s", ex.what());
 		Database::Destroy("ChatServer");
@@ -96,16 +91,11 @@ int main(int argc, char** argv) {
 	//Find out the master's IP:
 	std::string masterIP;
 	uint32_t masterPort = 1000;
-	sql::PreparedStatement* stmt = Database::CreatePreppedStmt("SELECT ip, port FROM servers WHERE name='master';");
-	auto res = stmt->executeQuery();
-	while (res->next()) {
-		masterIP = res->getString(1).c_str();
-		masterPort = res->getInt(2);
+	auto masterInfo = Database::Get()->GetMasterInfo();
+	if (masterInfo) {
+		masterIP = masterInfo->ip;
+		masterPort = masterInfo->port;
 	}
-
-	delete res;
-	delete stmt;
-
 	//It's safe to pass 'localhost' here, as the IP is only used as the external IP.
 	uint32_t maxClients = 50;
 	uint32_t ourPort = 1501;
@@ -158,15 +148,12 @@ int main(int argc, char** argv) {
 			//Find out the master's IP for absolutely no reason:
 			std::string masterIP;
 			uint32_t masterPort;
-			sql::PreparedStatement* stmt = Database::CreatePreppedStmt("SELECT ip, port FROM servers WHERE name='master';");
-			auto res = stmt->executeQuery();
-			while (res->next()) {
-				masterIP = res->getString(1).c_str();
-				masterPort = res->getInt(2);
-			}
 
-			delete res;
-			delete stmt;
+			auto masterInfo = Database::Get()->GetMasterInfo();
+			if (masterInfo) {
+				masterIP = masterInfo->ip;
+				masterPort = masterInfo->port;
+			}
 
 			framesSinceLastSQLPing = 0;
 		} else framesSinceLastSQLPing++;

dChatServer/PlayerContainer.cpp

@@ -48,14 +48,7 @@ void PlayerContainer::InsertPlayer(Packet* packet) {
 	m_Players.insert(std::make_pair(data->playerID, data));
 	LOG("Added user: %s (%llu), zone: %i", data->playerName.c_str(), data->playerID, data->zoneID.GetMapID());
 
-	auto* insertLog = Database::CreatePreppedStmt("INSERT INTO activity_log (character_id, activity, time, map_id) VALUES (?, ?, ?, ?);");
-
-	insertLog->setInt(1, data->playerID);
-	insertLog->setInt(2, 0);
-	insertLog->setUInt64(3, time(nullptr));
-	insertLog->setInt(4, data->zoneID.GetMapID());
-
-	insertLog->executeUpdate();
+	Database::Get()->UpdateActivityLog(data->playerID, eActivityType::PlayerLoggedIn, data->zoneID.GetMapID());
 }
 
 void PlayerContainer::RemovePlayer(Packet* packet) {
@@ -92,14 +85,7 @@ void PlayerContainer::RemovePlayer(Packet* packet) {
 	LOG("Removed user: %llu", playerID);
 	m_Players.erase(playerID);
 
-	auto* insertLog = Database::CreatePreppedStmt("INSERT INTO activity_log (character_id, activity, time, map_id) VALUES (?, ?, ?, ?);");
-
-	insertLog->setInt(1, playerID);
-	insertLog->setInt(2, 1);
-	insertLog->setUInt64(3, time(nullptr));
-	insertLog->setInt(4, player->zoneID.GetMapID());
-
-	insertLog->executeUpdate();
+	Database::Get()->UpdateActivityLog(playerID, eActivityType::PlayerLoggedOut, player->zoneID.GetMapID());
 }
 
 void PlayerContainer::MuteUpdate(Packet* packet) {

dCommon/BrickByBrickFix.cpp

@@ -13,9 +13,8 @@
 
 //! Forward declarations
 
-std::unique_ptr<sql::ResultSet> GetModelsFromDatabase();
 void WriteSd0Magic(char* input, uint32_t chunkSize);
-bool CheckSd0Magic(sql::Blob* streamToCheck);
+bool CheckSd0Magic(std::istream& streamToCheck);
 
 /**
  * @brief Truncates all models with broken data from the database.
@@ -24,28 +23,24 @@ bool CheckSd0Magic(sql::Blob* streamToCheck);
  */
 uint32_t BrickByBrickFix::TruncateBrokenBrickByBrickXml() {
 	uint32_t modelsTruncated{};
-	auto modelsToTruncate = GetModelsFromDatabase();
-	bool previousCommitValue = Database::GetAutoCommit();
-	Database::SetAutoCommit(false);
-	while (modelsToTruncate->next()) {
-		std::unique_ptr<sql::PreparedStatement> ugcModelToDelete(Database::CreatePreppedStmt("DELETE FROM ugc WHERE ugc.id = ?;"));
-		std::unique_ptr<sql::PreparedStatement> pcModelToDelete(Database::CreatePreppedStmt("DELETE FROM properties_contents WHERE ugc_id = ?;"));
+	auto modelsToTruncate = Database::Get()->GetAllUgcModels();
+	bool previousCommitValue = Database::Get()->GetAutoCommit();
+	Database::Get()->SetAutoCommit(false);
+	for (auto& model : modelsToTruncate) {
 		std::string completeUncompressedModel{};
 		uint32_t chunkCount{};
-		uint64_t modelId = modelsToTruncate->getInt(1);
-		std::unique_ptr<sql::Blob> modelAsSd0(modelsToTruncate->getBlob(2));
 		// Check that header is sd0 by checking for the sd0 magic.
-		if (CheckSd0Magic(modelAsSd0.get())) {
+		if (CheckSd0Magic(model.lxfmlData)) {
 			while (true) {
 				uint32_t chunkSize{};
-				modelAsSd0->read(reinterpret_cast<char*>(&chunkSize), sizeof(uint32_t)); // Extract chunk size from istream
+				model.lxfmlData.read(reinterpret_cast<char*>(&chunkSize), sizeof(uint32_t)); // Extract chunk size from istream
 
 				// Check if good here since if at the end of an sd0 file, this will have eof flagged.
-				if (!modelAsSd0->good()) break;
+				if (!model.lxfmlData.good()) break;
 
 				std::unique_ptr<uint8_t[]> compressedChunk(new uint8_t[chunkSize]);
 				for (uint32_t i = 0; i < chunkSize; i++) {
-					compressedChunk[i] = modelAsSd0->get();
+					compressedChunk[i] = model.lxfmlData.get();
 				}
 
 				// Ignore the valgrind warning about uninitialized values.  These are discarded later when we know the actual uncompressed size.
@@ -59,7 +54,7 @@ uint32_t BrickByBrickFix::TruncateBrokenBrickByBrickXml() {
 					completeUncompressedModel.append((char*)uncompressedChunk.get());
 					completeUncompressedModel.resize(previousSize + actualUncompressedSize);
 				} else {
-					LOG("Failed to inflate chunk %i for model %llu.  Error: %i", chunkCount, modelId, err);
+					LOG("Failed to inflate chunk %i for model %llu.  Error: %i", chunkCount, model.id, err);
 					break;
 				}
 				chunkCount++;
@@ -75,26 +70,20 @@ uint32_t BrickByBrickFix::TruncateBrokenBrickByBrickXml() {
 					"</LXFML>",
 					completeUncompressedModel.length() >= 15 ? completeUncompressedModel.length() - 15 : 0) == std::string::npos
 					) {
-					LOG("Brick-by-brick model %llu will be deleted!", modelId);
-					ugcModelToDelete->setInt64(1, modelsToTruncate->getInt64(1));
-					pcModelToDelete->setInt64(1, modelsToTruncate->getInt64(1));
-					ugcModelToDelete->execute();
-					pcModelToDelete->execute();
+					LOG("Brick-by-brick model %llu will be deleted!", model.id);
+					Database::Get()->DeleteUgcModelData(model.id);
 					modelsTruncated++;
 				}
 			}
 		} else {
-			LOG("Brick-by-brick model %llu will be deleted!", modelId);
-			ugcModelToDelete->setInt64(1, modelsToTruncate->getInt64(1));
-			pcModelToDelete->setInt64(1, modelsToTruncate->getInt64(1));
-			ugcModelToDelete->execute();
-			pcModelToDelete->execute();
+			LOG("Brick-by-brick model %llu will be deleted!", model.id);
+			Database::Get()->DeleteUgcModelData(model.id);
 			modelsTruncated++;
 		}
 	}
 
-	Database::Commit();
-	Database::SetAutoCommit(previousCommitValue);
+	Database::Get()->Commit();
+	Database::Get()->SetAutoCommit(previousCommitValue);
 	return modelsTruncated;
 }
 
@@ -106,21 +95,17 @@ uint32_t BrickByBrickFix::TruncateBrokenBrickByBrickXml() {
  */
 uint32_t BrickByBrickFix::UpdateBrickByBrickModelsToSd0() {
 	uint32_t updatedModels = 0;
-	auto modelsToUpdate = GetModelsFromDatabase();
-	auto previousAutoCommitState = Database::GetAutoCommit();
-	Database::SetAutoCommit(false);
-	std::unique_ptr<sql::PreparedStatement> insertionStatement(Database::CreatePreppedStmt("UPDATE ugc SET lxfml = ? WHERE id = ?;"));
-	while (modelsToUpdate->next()) {
-		int64_t modelId = modelsToUpdate->getInt64(1);
-		std::unique_ptr<sql::Blob> oldLxfml(modelsToUpdate->getBlob(2));
+	auto modelsToUpdate = Database::Get()->GetAllUgcModels();
+	auto previousAutoCommitState = Database::Get()->GetAutoCommit();
+	Database::Get()->SetAutoCommit(false);
+	for (auto& model : modelsToUpdate) {
 		// Check if the stored blob starts with zlib magic (0x78 0xDA - best compression of zlib)
 		// If it does, convert it to sd0.
-		if (oldLxfml->get() == 0x78 && oldLxfml->get() == 0xDA) {
-
+		if (model.lxfmlData.get() == 0x78 && model.lxfmlData.get() == 0xDA) {
 			// Get and save size of zlib compressed chunk.
-			oldLxfml->seekg(0, std::ios::end);
-			uint32_t oldLxfmlSize = static_cast<uint32_t>(oldLxfml->tellg());
-			oldLxfml->seekg(0);
+			model.lxfmlData.seekg(0, std::ios::end);
+			uint32_t oldLxfmlSize = static_cast<uint32_t>(model.lxfmlData.tellg());
+			model.lxfmlData.seekg(0);
 
 			// Allocate 9 extra bytes.  5 for sd0 magic, 4 for the only zlib compressed size.
 			uint32_t oldLxfmlSizeWithHeader = oldLxfmlSize + 9;
@@ -128,34 +113,27 @@ uint32_t BrickByBrickFix::UpdateBrickByBrickModelsToSd0() {
 
 			WriteSd0Magic(sd0ConvertedModel.get(), oldLxfmlSize);
 			for (uint32_t i = 9; i < oldLxfmlSizeWithHeader; i++) {
-				sd0ConvertedModel.get()[i] = oldLxfml->get();
+				sd0ConvertedModel.get()[i] = model.lxfmlData.get();
 			}
 
 			std::string outputString(sd0ConvertedModel.get(), oldLxfmlSizeWithHeader);
 			std::istringstream outputStringStream(outputString);
 
-			insertionStatement->setBlob(1, static_cast<std::istream*>(&outputStringStream));
-			insertionStatement->setInt64(2, modelId);
 			try {
-				insertionStatement->executeUpdate();
-				LOG("Updated model %i to sd0", modelId);
+				Database::Get()->UpdateUgcModelData(model.id, outputStringStream);
+				LOG("Updated model %i to sd0", model.id);
 				updatedModels++;
 			} catch (sql::SQLException exception) {
 				LOG("Failed to update model %i.  This model should be inspected manually to see why."
-					"The database error is %s", modelId, exception.what());
+					"The database error is %s", model.id, exception.what());
 			}
 		}
 	}
-	Database::Commit();
-	Database::SetAutoCommit(previousAutoCommitState);
+	Database::Get()->Commit();
+	Database::Get()->SetAutoCommit(previousAutoCommitState);
 	return updatedModels;
 }
 
-std::unique_ptr<sql::ResultSet> GetModelsFromDatabase() {
-	std::unique_ptr<sql::PreparedStatement> modelsRawDataQuery(Database::CreatePreppedStmt("SELECT id, lxfml FROM ugc;"));
-	return std::unique_ptr<sql::ResultSet>(modelsRawDataQuery->executeQuery());
-}
-
 /**
  * @brief Writes sd0 magic at the front of a char*
  *
@@ -171,6 +149,6 @@ void WriteSd0Magic(char* input, uint32_t chunkSize) {
 	*reinterpret_cast<uint32_t*>(input + 5) = chunkSize; // Write the integer to the character array
 }
 
-bool CheckSd0Magic(sql::Blob* streamToCheck) {
-	return streamToCheck->get() == 's' && streamToCheck->get() == 'd' && streamToCheck->get() == '0' && streamToCheck->get() == 0x01 && streamToCheck->get() == 0xFF;
+bool CheckSd0Magic(std::istream& streamToCheck) {
+	return streamToCheck.get() == 's' && streamToCheck.get() == 'd' && streamToCheck.get() == '0' && streamToCheck.get() == 0x01 && streamToCheck.get() == 0xFF;
 }

dDatabase/CDClientDatabase/CDClientTables/CDComponentsRegistryTable.h

@@ -3,6 +3,8 @@
 // Custom Classes
 #include "CDTable.h"
 
+#include <unordered_map>
+
 enum class eReplicaComponentType : uint32_t;
 struct CDComponentsRegistry {
 	unsigned int id;                    //!< The LOT is used as the ID

dDatabase/CDClientDatabase/CDClientTables/CMakeLists.txt

@@ -1,4 +1,4 @@
-set(DDATABASE_TABLES_SOURCES "CDActivitiesTable.cpp"
+set(DDATABASE_CDCLIENTDATABASE_CDCLIENTTABLES_SOURCES "CDActivitiesTable.cpp"
 	"CDActivityRewardsTable.cpp"
 	"CDAnimationsTable.cpp"
 	"CDBehaviorParameterTable.cpp"

dDatabase/CDClientDatabase/CMakeLists.txt

@@ -0,0 +1,12 @@
+set(DDATABASE_CDCLIENTDATABASE_SOURCES
+	"CDClientDatabase.cpp"
+	"CDClientManager.cpp"
+)
+
+add_subdirectory(CDClientTables)
+
+foreach(file ${DDATABASE_CDCLIENTDATABASE_CDCLIENTTABLES_SOURCES})
+	set(DDATABASE_CDCLIENTDATABASE_SOURCES ${DDATABASE_CDCLIENTDATABASE_SOURCES} "CDClientTables/${file}")
+endforeach()
+
+set(DDATABASE_CDCLIENTDATABASE_SOURCES ${DDATABASE_CDCLIENTDATABASE_SOURCES} PARENT_SCOPE)

dDatabase/CMakeLists.txt

@@ -1,12 +1,15 @@
-set(DDATABASE_SOURCES "CDClientDatabase.cpp"
-		"CDClientManager.cpp"
-		"Database.cpp"
-		"MigrationRunner.cpp")
+set(DDATABASE_SOURCES)
 
-add_subdirectory(Tables)
+add_subdirectory(CDClientDatabase)
 
-foreach(file ${DDATABASE_TABLES_SOURCES})
-	set(DDATABASE_SOURCES ${DDATABASE_SOURCES} "Tables/${file}")
+foreach(file ${DDATABASE_CDCLIENTDATABASE_SOURCES})
+	set(DDATABASE_SOURCES ${DDATABASE_SOURCES} "CDClientDatabase/${file}")
+endforeach()
+
+add_subdirectory(GameDatabase)
+
+foreach(file ${DDATABASE_GAMEDATABASE_SOURCES})
+	set(DDATABASE_SOURCES ${DDATABASE_SOURCES} "GameDatabase/${file}")
 endforeach()
 
 add_library(dDatabase STATIC ${DDATABASE_SOURCES})

dDatabase/Database.cpp

@@ -1,118 +0,0 @@
-#include "Database.h"
-#include "Game.h"
-#include "dConfig.h"
-#include "Logger.h"
-using namespace std;
-
-#pragma warning (disable:4251) //Disables SQL warnings
-
-sql::Driver* Database::driver;
-sql::Connection* Database::con;
-sql::Properties Database::props;
-std::string Database::database;
-
-void Database::Connect(const string& host, const string& database, const string& username, const string& password) {
-
-	//To bypass debug issues:
-	const char* szDatabase = database.c_str();
-	const char* szUsername = username.c_str();
-	const char* szPassword = password.c_str();
-
-	driver = sql::mariadb::get_driver_instance();
-
-	sql::Properties properties;
-	// The mariadb connector is *supposed* to handle unix:// and pipe:// prefixes to hostName, but there are bugs where
-	// 1) it tries to parse a database from the connection string (like in tcp://localhost:3001/darkflame) based on the
-	//    presence of a /
-	// 2) even avoiding that, the connector still assumes you're connecting with a tcp socket
-	// So, what we do in the presence of a unix socket or pipe is to set the hostname to the protocol and localhost,
-	// which avoids parsing errors while still ensuring the correct connection type is used, and then setting the appropriate
-	// property manually (which the URL parsing fails to do)
-	const std::string UNIX_PROTO = "unix://";
-	const std::string PIPE_PROTO = "pipe://";
-    if (host.find(UNIX_PROTO) == 0) {
-		properties["hostName"] = "unix://localhost";
-		properties["localSocket"] = host.substr(UNIX_PROTO.length()).c_str();
-    } else if (host.find(PIPE_PROTO) == 0) {
-		properties["hostName"] = "pipe://localhost";
-		properties["pipe"] = host.substr(PIPE_PROTO.length()).c_str();
-    } else {
-		properties["hostName"] = host.c_str();
-    }
-	properties["user"] = szUsername;
-	properties["password"] = szPassword;
-	properties["autoReconnect"] = "true";
-
-	Database::props = properties;
-	Database::database = database;
-
-	Database::Connect();
-}
-
-void Database::Connect() {
-	// `connect(const Properties& props)` segfaults in windows debug, but
-	// `connect(const SQLString& host, const SQLString& user, const SQLString& pwd)` doesn't handle pipes/unix sockets correctly
-	if (Database::props.find("localSocket") != Database::props.end() || Database::props.find("pipe") != Database::props.end()) {
-		con = driver->connect(Database::props);
-	} else {
-		con = driver->connect(Database::props["hostName"].c_str(), Database::props["user"].c_str(), Database::props["password"].c_str());
-	}
-	con->setSchema(Database::database.c_str());
-}
-
-void Database::Destroy(std::string source, bool log) {
-	if (!con) return;
-
-	if (log) {
-		if (source != "") LOG("Destroying MySQL connection from %s!", source.c_str());
-		else LOG("Destroying MySQL connection!");
-	}
-
-	con->close();
-	delete con;
-} //Destroy
-
-sql::Statement* Database::CreateStmt() {
-	sql::Statement* toReturn = con->createStatement();
-	return toReturn;
-} //CreateStmt
-
-sql::PreparedStatement* Database::CreatePreppedStmt(const std::string& query) {
-	const char* test = query.c_str();
-	size_t size = query.length();
-	sql::SQLString str(test, size);
-
-	if (!con) {
-		Connect();
-		LOG("Trying to reconnect to MySQL");
-	}
-
-	if (!con->isValid() || con->isClosed()) {
-		delete con;
-
-		con = nullptr;
-
-		Connect();
-		LOG("Trying to reconnect to MySQL from invalid or closed connection");
-	}
-
-	auto* stmt = con->prepareStatement(str);
-
-	return stmt;
-} //CreatePreppedStmt
-
-void Database::Commit() {
-	Database::con->commit();
-}
-
-bool Database::GetAutoCommit() {
-	// TODO This should not just access a pointer.  A future PR should update this
-	// to check for null and throw an error if the connection is not valid.
-	return con->getAutoCommit();
-}
-
-void Database::SetAutoCommit(bool value) {
-	// TODO This should not just access a pointer.  A future PR should update this
-	// to check for null and throw an error if the connection is not valid.
-	Database::con->setAutoCommit(value);
-}

dDatabase/Database.h

@@ -1,31 +0,0 @@
-#pragma once
-
-#include <string>
-#include <conncpp.hpp>
-
-class MySqlException : public std::runtime_error {
-public:
-	MySqlException() : std::runtime_error("MySQL error!") {}
-	MySqlException(const std::string& msg) : std::runtime_error(msg.c_str()) {}
-};
-
-class Database {
-private:
-	static sql::Driver* driver;
-	static sql::Connection* con;
-	static sql::Properties props;
-	static std::string database;
-public:
-	static void Connect(const std::string& host, const std::string& database, const std::string& username, const std::string& password);
-	static void Connect();
-	static void Destroy(std::string source = "", bool log = true);
-
-	static sql::Statement* CreateStmt();
-	static sql::PreparedStatement* CreatePreppedStmt(const std::string& query);
-	static void Commit();
-	static bool GetAutoCommit();
-	static void SetAutoCommit(bool value);
-
-	static std::string GetDatabase() { return database; }
-	static sql::Properties GetProperties() { return props; }
-};

dDatabase/GameDatabase/CMakeLists.txt

@@ -0,0 +1,12 @@
+set(DDATABASE_GAMEDATABASE_SOURCES
+	"Database.cpp"
+	"MigrationRunner.cpp"
+)
+
+add_subdirectory(MySQL)
+
+foreach(file ${DDATABSE_DATABSES_MYSQL_SOURCES})
+	set(DDATABASE_GAMEDATABASE_SOURCES ${DDATABASE_GAMEDATABASE_SOURCES} "MySQL/${file}")
+endforeach()
+
+set(DDATABASE_GAMEDATABASE_SOURCES ${DDATABASE_GAMEDATABASE_SOURCES} PARENT_SCOPE)

dDatabase/GameDatabase/Database.cpp

@@ -0,0 +1,40 @@
+#include "Database.h"
+#include "Game.h"
+#include "dConfig.h"
+#include "Logger.h"
+#include "MySQLDatabase.h"
+#include "DluAssert.h"
+
+#pragma warning (disable:4251) //Disables SQL warnings
+
+namespace {
+	GameDatabase* database = nullptr;
+}
+
+void Database::Connect() {
+	if (database) {
+		LOG("Tried to connect to database when it's already connected!");
+		return;
+	}
+
+	database = new MySQLDatabase();
+	database->Connect();
+}
+
+GameDatabase* Database::Get() {
+	if (!database) {
+		LOG("Tried to get database when it's not connected!");
+		Connect();
+	}
+	return database;
+}
+
+void Database::Destroy(std::string source) {
+	if (database) {
+		database->Destroy(source);
+		delete database;
+		database = nullptr;
+	} else {
+		LOG("Trying to destroy database when it's not connected!");
+	}
+}

dDatabase/GameDatabase/Database.h

@@ -0,0 +1,12 @@
+#pragma once
+
+#include <string>
+#include <conncpp.hpp>
+
+#include "GameDatabase.h"
+
+namespace Database {
+	void Connect();
+	GameDatabase* Get();
+	void Destroy(std::string source = "");
+};

dDatabase/GameDatabase/GameDatabase.h

@@ -0,0 +1,55 @@
+#ifndef __GAMEDATABASE__H__
+#define __GAMEDATABASE__H__
+
+#include <optional>
+
+#include "ILeaderboard.h"
+#include "IPlayerCheatDetections.h"
+#include "ICommandLog.h"
+#include "IMail.h"
+#include "IObjectIdTracker.h"
+#include "IPlayKeys.h"
+#include "IServers.h"
+#include "IBugReports.h"
+#include "IPropertyContents.h"
+#include "IProperty.h"
+#include "IPetNames.h"
+#include "ICharXml.h"
+#include "IMigrationHistory.h"
+#include "IUgc.h"
+#include "IFriends.h"
+#include "ICharInfo.h"
+#include "IAccounts.h"
+#include "IActivityLog.h"
+
+namespace sql {
+	class Statement;
+	class PreparedStatement;
+};
+
+#ifdef _DEBUG
+#  define DLU_SQL_TRY_CATCH_RETHROW(x) do { try { x; } catch (sql::SQLException& ex) { LOG("SQL Error: %s", ex.what()); throw; } } while(0)
+#else
+#  define DLU_SQL_TRY_CATCH_RETHROW(x) x
+#endif // _DEBUG
+
+class GameDatabase :
+	public IPlayKeys, public ILeaderboard, public IObjectIdTracker, public IServers,
+	public IMail, public ICommandLog, public IPlayerCheatDetections, public IBugReports,
+	public IPropertyContents, public IProperty, public IPetNames, public ICharXml,
+	public IMigrationHistory, public IUgc, public IFriends, public ICharInfo,
+	public IAccounts, public IActivityLog {
+public:
+	virtual ~GameDatabase() = default;
+	// TODO: These should be made private.
+	virtual void Connect() = 0;
+	virtual void Destroy(std::string source = "") = 0;
+	virtual void ExecuteCustomQuery(const std::string_view query) = 0;
+	virtual sql::PreparedStatement* CreatePreppedStmt(const std::string& query) = 0;
+	virtual void Commit() = 0;
+	virtual bool GetAutoCommit() = 0;
+	virtual void SetAutoCommit(bool value) = 0;
+	virtual void DeleteCharacter(const uint32_t characterId) = 0;
+};
+
+#endif  //!__GAMEDATABASE__H__

dDatabase/GameDatabase/ITables/IAccounts.h

@@ -0,0 +1,37 @@
+#ifndef __IACCOUNTS__H__
+#define __IACCOUNTS__H__
+
+#include <cstdint>
+#include <optional>
+#include <string_view>
+
+enum class eGameMasterLevel : uint8_t;
+
+class IAccounts {
+public:
+	struct Info {
+		std::string bcryptPassword;
+		uint32_t id{};
+		uint32_t playKeyId{};
+		bool banned{};
+		bool locked{};
+		eGameMasterLevel maxGmLevel{};
+	};
+
+	// Get the account info for the given username.
+	virtual std::optional<IAccounts::Info> GetAccountInfo(const std::string_view username) = 0;
+
+	// Update the account's unmute time.
+	virtual void UpdateAccountUnmuteTime(const uint32_t accountId, const uint64_t timeToUnmute) = 0;
+
+	// Update the account's ban status.
+	virtual void UpdateAccountBan(const uint32_t accountId, const bool banned) = 0;
+
+	// Update the account's password.
+	virtual void UpdateAccountPassword(const uint32_t accountId, const std::string_view bcryptpassword) = 0;
+
+	// Add a new account to the database.
+	virtual void InsertNewAccount(const std::string_view username, const std::string_view bcryptpassword) = 0;
+};
+
+#endif  //!__IACCOUNTS__H__

dDatabase/GameDatabase/ITables/IActivityLog.h

@@ -0,0 +1,19 @@
+#ifndef __IACTIVITYLOG__H__
+#define __IACTIVITYLOG__H__
+
+#include <cstdint>
+
+#include "dCommonVars.h"
+
+enum class eActivityType : uint32_t {
+	PlayerLoggedIn,
+	PlayerLoggedOut,
+};
+
+class IActivityLog {
+public:
+	// Update the activity log for the given account.
+	virtual void UpdateActivityLog(const uint32_t characterId, const eActivityType activityType, const LWOMAPID mapId) = 0;
+};
+
+#endif  //!__IACTIVITYLOG__H__

dDatabase/GameDatabase/ITables/IBugReports.h

@@ -0,0 +1,20 @@
+#ifndef __IBUGREPORTS__H__
+#define __IBUGREPORTS__H__
+
+#include <cstdint>
+#include <string_view>
+
+class IBugReports {
+public:
+	struct Info {
+		std::string body;
+		std::string clientVersion;
+		std::string otherPlayer;
+		std::string selection;
+		uint32_t characterId{};
+	};
+
+	// Add a new bug report to the database.
+	virtual void InsertNewBugReport(const Info& info) = 0;
+};
+#endif  //!__IBUGREPORTS__H__