refactor: Database abstraction and organization of files (#1274)

* Database: Convert to proper namespace

* Database: Use base class and getter

* Database: Move files around

* Database: Add property Management query

Database: Move over user queries

Tested at gm 0 that pre-approved names are pre-approved, unapproved need moderator approval
deleting characters deletes the selcted one
refreshing the character page shows the last character you logged in as
tested all my characters show up when i login
tested that you can delete all 4 characters and the correct character is selected each time
tested renaming, approving names as gm0

Database: Add ugc model getter

Hey it works, look I got around the mariadb issue.

Database: Add queries

Database: consolidate name query

Database: Add friends list query

Update name of approved names query

Documentation

Database: Add name check

Database: Add BFF Query

Database: Move BFF Setter

Database: Move new friend query

Database: Add remove friend queries

Database: Add activity log

Database: Add ugc & prop content removal

Database: Add model update

Database: Add migration queries

Database: Add character and xml queries

Database: Add user queries

Untested, but compiling code

Need to test that new character names are properly assigned in the following scenarios
gm 0 and pre-approved name
gm 0 and unapproved name
gm 9 and pre-approved name
gm 9 and unapproved name

Database: constify function arguments

Database: Add pet queries

* Database: Move property model queries

Untested.  Need to test
placing a new model
moving existing one
removing ugc model
placing ugc model
moving ugc model(?)
changing privacy option variously
change description and name
approve property
can properly travel to property

* Property: Move stale reference deletion

* Database: Move performance update query

* Database: Add bug report query

* Database: Add cheat detection query

* Database: Add mail send query

* Untested code

need to test mailing from slash command, from all users of SendMail, getting bbb of a property and sending messages to bffs

* Update CDComponentsRegistryTable.h

Database: Rename and add further comments

Datavbase: Add comments

Add some comments

Build: Fix PCH directories

Database: Fix time

thanks apple

Database: Fix compiler warnings

Overload destructor
Define specialty for time_t
Use string instead of string_view for temp empty string

Update CDTable.h

Property: Update queries to use mapId

Database: Reorganize

Reorganize into CDClient folder and GameDatabase folder for clearer meanings and file structure

Folders: Rename to GameDatabase

MySQL: Remove MySQL Specifier from table

Database: Move Tables to Interfaces

Database: Reorder functions in header

Database: Simplify property queries

Database: Remove unused queries

Remove extra query definitions as well

Database: Consolidate User getters

Database: Comment logs

Update MySQLDatabase.cpp

Database: Use generic code

Playkey: Fix bad optional access

Database: Move stuff around

WorldServer: Update queries

Ugc reduced by many scopes
use new queries
very fast
tested that ugc still loads

Database: Add auth queries

I tested that only the correct password can sign into an account.
Tested that disabled playkeys do not allow the user to play the game

Database: Add donation query

Database: add objectId queries

Database: Add master queries

Database: Fix mis-named function

Database: Add slash command queries

Mail: Fix itemId type

CharFilter: Use new query

ObjectID: Remove duplicate code

SlashCommand: Update query with function

Database: Add mail queries

Ugc: Fix issues with saving models

Resolve large scope blocks as well

* Database: Add debug try catch rethrow macro

* General fixes

* fix play key not working

* Further fixes

---------

Co-authored-by: Aaron Kimbre <aronwk.aaron@gmail.com>
This commit is contained in:
David Markowitz 2023-11-17 16:47:18 -08:00 committed by GitHub
parent b68823b4cb
commit 7f623d358c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
161 changed files with 2114 additions and 1516 deletions

View File

@ -214,7 +214,12 @@ set(INCLUDED_DIRECTORIES
"dNavigation/dTerrain" "dNavigation/dTerrain"
"dZoneManager" "dZoneManager"
"dDatabase" "dDatabase"
"dDatabase/Tables" "dDatabase/CDClientDatabase"
"dDatabase/CDClientDatabase/CDClientTables"
"dDatabase/GameDatabase"
"dDatabase/GameDatabase/ITables"
"dDatabase/GameDatabase/MySQL"
"dDatabase/GameDatabase/MySQL/Tables"
"dNet" "dNet"
"dScripts" "dScripts"
"dScripts/02_server" "dScripts/02_server"
@ -329,8 +334,9 @@ add_subdirectory(thirdparty)
file( file(
GLOB HEADERS_DDATABASE GLOB HEADERS_DDATABASE
LIST_DIRECTORIES false LIST_DIRECTORIES false
${PROJECT_SOURCE_DIR}/dDatabase/*.h ${PROJECT_SOURCE_DIR}/dDatabase/CDClientDatabase/*.h
${PROJECT_SOURCE_DIR}/dDatabase/Tables/*.h ${PROJECT_SOURCE_DIR}/dDatabase/CDClientDatabase/CDClientTables/*.h
${PROJECT_SOURCE_DIR}/dDatabase/GameDatabase/ITables/*.h
${PROJECT_SOURCE_DIR}/thirdparty/SQLite/*.h ${PROJECT_SOURCE_DIR}/thirdparty/SQLite/*.h
) )

View File

@ -55,14 +55,8 @@ int main(int argc, char** argv) {
LOG("Version: %i.%i", PROJECT_VERSION_MAJOR, PROJECT_VERSION_MINOR); LOG("Version: %i.%i", PROJECT_VERSION_MAJOR, PROJECT_VERSION_MINOR);
LOG("Compiled on: %s", __TIMESTAMP__); LOG("Compiled on: %s", __TIMESTAMP__);
//Connect to the MySQL Database
std::string mysql_host = Game::config->GetValue("mysql_host");
std::string mysql_database = Game::config->GetValue("mysql_database");
std::string mysql_username = Game::config->GetValue("mysql_username");
std::string mysql_password = Game::config->GetValue("mysql_password");
try { try {
Database::Connect(mysql_host, mysql_database, mysql_username, mysql_password); Database::Connect();
} catch (sql::SQLException& ex) { } catch (sql::SQLException& ex) {
LOG("Got an error while connecting to the database: %s", ex.what()); LOG("Got an error while connecting to the database: %s", ex.what());
Database::Destroy("AuthServer"); Database::Destroy("AuthServer");
@ -74,15 +68,12 @@ int main(int argc, char** argv) {
//Find out the master's IP: //Find out the master's IP:
std::string masterIP; std::string masterIP;
uint32_t masterPort = 1500; uint32_t masterPort = 1500;
sql::PreparedStatement* stmt = Database::CreatePreppedStmt("SELECT ip, port FROM servers WHERE name='master';");
auto res = stmt->executeQuery();
while (res->next()) {
masterIP = res->getString(1).c_str();
masterPort = res->getInt(2);
}
delete res; auto masterInfo = Database::Get()->GetMasterInfo();
delete stmt; if (masterInfo) {
masterIP = masterInfo->ip;
masterPort = masterInfo->port;
}
Game::randomEngine = std::mt19937(time(0)); Game::randomEngine = std::mt19937(time(0));
@ -134,16 +125,12 @@ int main(int argc, char** argv) {
//Find out the master's IP for absolutely no reason: //Find out the master's IP for absolutely no reason:
std::string masterIP; std::string masterIP;
uint32_t masterPort; uint32_t masterPort;
sql::PreparedStatement* stmt = Database::CreatePreppedStmt("SELECT ip, port FROM servers WHERE name='master';"); auto masterInfo = Database::Get()->GetMasterInfo();
auto res = stmt->executeQuery(); if (masterInfo) {
while (res->next()) { masterIP = masterInfo->ip;
masterIP = res->getString(1).c_str(); masterPort = masterInfo->port;
masterPort = res->getInt(2);
} }
delete res;
delete stmt;
framesSinceLastSQLPing = 0; framesSinceLastSQLPing = 0;
} else framesSinceLastSQLPing++; } else framesSinceLastSQLPing++;

View File

@ -32,15 +32,11 @@ dChatFilter::dChatFilter(const std::string& filepath, bool dontGenerateDCF) {
} }
//Read player names that are ok as well: //Read player names that are ok as well:
auto stmt = Database::CreatePreppedStmt("select name from charinfo;"); auto approvedNames = Database::Get()->GetApprovedCharacterNames();
auto res = stmt->executeQuery(); for (auto& name : approvedNames) {
while (res->next()) { std::transform(name.begin(), name.end(), name.begin(), ::tolower); //Transform to lowercase
std::string line = res->getString(1).c_str(); m_ApprovedWords.push_back(CalculateHash(name));
std::transform(line.begin(), line.end(), line.begin(), ::tolower); //Transform to lowercase
m_ApprovedWords.push_back(CalculateHash(line));
} }
delete res;
delete stmt;
} }
dChatFilter::~dChatFilter() { dChatFilter::~dChatFilter() {

View File

@ -30,32 +30,17 @@ void ChatPacketHandler::HandleFriendlistRequest(Packet* packet) {
auto player = playerContainer.GetPlayerData(playerID); auto player = playerContainer.GetPlayerData(playerID);
if (!player) return; if (!player) return;
//Get our friends list from the Db. Using a derived table since the friend of a player can be in either column. auto friendsList = Database::Get()->GetFriendsList(playerID);
std::unique_ptr<sql::PreparedStatement> stmt(Database::CreatePreppedStmt( for (const auto& friendData : friendsList) {
"SELECT fr.requested_player, best_friend, ci.name FROM "
"(SELECT CASE "
"WHEN player_id = ? THEN friend_id "
"WHEN friend_id = ? THEN player_id "
"END AS requested_player, best_friend FROM friends) AS fr "
"JOIN charinfo AS ci ON ci.id = fr.requested_player "
"WHERE fr.requested_player IS NOT NULL AND fr.requested_player != ?;"));
stmt->setUInt(1, static_cast<uint32_t>(playerID));
stmt->setUInt(2, static_cast<uint32_t>(playerID));
stmt->setUInt(3, static_cast<uint32_t>(playerID));
std::vector<FriendData> friends;
std::unique_ptr<sql::ResultSet> res(stmt->executeQuery());
while (res->next()) {
FriendData fd; FriendData fd;
fd.isFTP = false; // not a thing in DLU fd.isFTP = false; // not a thing in DLU
fd.friendID = res->getUInt(1); fd.friendID = friendData.friendID;
GeneralUtils::SetBit(fd.friendID, eObjectBits::PERSISTENT); GeneralUtils::SetBit(fd.friendID, eObjectBits::PERSISTENT);
GeneralUtils::SetBit(fd.friendID, eObjectBits::CHARACTER); GeneralUtils::SetBit(fd.friendID, eObjectBits::CHARACTER);
fd.isBestFriend = res->getInt(2) == 3; //0 = friends, 1 = left_requested, 2 = right_requested, 3 = both_accepted - are now bffs fd.isBestFriend = friendData.isBestFriend; //0 = friends, 1 = left_requested, 2 = right_requested, 3 = both_accepted - are now bffs
if (fd.isBestFriend) player->countOfBestFriends += 1; if (fd.isBestFriend) player->countOfBestFriends += 1;
fd.friendName = res->getString(3); fd.friendName = friendData.friendName;
//Now check if they're online: //Now check if they're online:
auto fr = playerContainer.GetPlayerData(fd.friendID); auto fr = playerContainer.GetPlayerData(fd.friendID);
@ -71,7 +56,7 @@ void ChatPacketHandler::HandleFriendlistRequest(Packet* packet) {
fd.zoneID = LWOZONEID(); fd.zoneID = LWOZONEID();
} }
friends.push_back(fd); player->friends.push_back(fd);
} }
//Now, we need to send the friendlist to the server they came from: //Now, we need to send the friendlist to the server they came from:
@ -83,14 +68,12 @@ void ChatPacketHandler::HandleFriendlistRequest(Packet* packet) {
BitStreamUtils::WriteHeader(bitStream, eConnectionType::CLIENT, eClientMessageType::GET_FRIENDS_LIST_RESPONSE); BitStreamUtils::WriteHeader(bitStream, eConnectionType::CLIENT, eClientMessageType::GET_FRIENDS_LIST_RESPONSE);
bitStream.Write<uint8_t>(0); bitStream.Write<uint8_t>(0);
bitStream.Write<uint16_t>(1); //Length of packet -- just writing one as it doesn't matter, client skips it. bitStream.Write<uint16_t>(1); //Length of packet -- just writing one as it doesn't matter, client skips it.
bitStream.Write((uint16_t)friends.size()); bitStream.Write((uint16_t)player->friends.size());
for (auto& data : friends) { for (auto& data : player->friends) {
data.Serialize(bitStream); data.Serialize(bitStream);
} }
player->friends = friends;
SystemAddress sysAddr = player->sysAddr; SystemAddress sysAddr = player->sysAddr;
SEND_PACKET; SEND_PACKET;
} }
@ -152,35 +135,26 @@ void ChatPacketHandler::HandleFriendRequest(Packet* packet) {
// If at this point we dont have a target, then they arent online and we cant send the request. // If at this point we dont have a target, then they arent online and we cant send the request.
// Send the response code that corresponds to what the error is. // Send the response code that corresponds to what the error is.
if (!requestee) { if (!requestee) {
std::unique_ptr<sql::PreparedStatement> nameQuery(Database::CreatePreppedStmt("SELECT name from charinfo where name = ?;"));
nameQuery->setString(1, playerName);
std::unique_ptr<sql::ResultSet> result(nameQuery->executeQuery());
requestee.reset(new PlayerData()); requestee.reset(new PlayerData());
requestee->playerName = playerName; requestee->playerName = playerName;
auto responseType = Database::Get()->GetCharacterInfo(playerName)
? eAddFriendResponseType::NOTONLINE
: eAddFriendResponseType::INVALIDCHARACTER;
SendFriendResponse(requestor, requestee.get(), result->next() ? eAddFriendResponseType::NOTONLINE : eAddFriendResponseType::INVALIDCHARACTER); SendFriendResponse(requestor, requestee.get(), responseType);
return; return;
} }
if (isBestFriendRequest) { if (isBestFriendRequest) {
std::unique_ptr<sql::PreparedStatement> friendUpdate(Database::CreatePreppedStmt("SELECT * FROM friends WHERE (player_id = ? AND friend_id = ?) OR (player_id = ? AND friend_id = ?) LIMIT 1;"));
friendUpdate->setUInt(1, static_cast<uint32_t>(requestorPlayerID));
friendUpdate->setUInt(2, static_cast<uint32_t>(requestee->playerID));
friendUpdate->setUInt(3, static_cast<uint32_t>(requestee->playerID));
friendUpdate->setUInt(4, static_cast<uint32_t>(requestorPlayerID));
std::unique_ptr<sql::ResultSet> result(friendUpdate->executeQuery());
LWOOBJID queryPlayerID = LWOOBJID_EMPTY;
LWOOBJID queryFriendID = LWOOBJID_EMPTY;
uint8_t oldBestFriendStatus{}; uint8_t oldBestFriendStatus{};
uint8_t bestFriendStatus{}; uint8_t bestFriendStatus{};
auto bestFriendInfo = Database::Get()->GetBestFriendStatus(requestorPlayerID, requestee->playerID);
if (result->next()) { if (bestFriendInfo) {
// Get the IDs // Get the IDs
queryPlayerID = result->getInt(1); LWOOBJID queryPlayerID = bestFriendInfo->playerCharacterId;
queryFriendID = result->getInt(2); LWOOBJID queryFriendID = bestFriendInfo->friendCharacterId;
oldBestFriendStatus = result->getInt(3); oldBestFriendStatus = bestFriendInfo->bestFriendStatus;
bestFriendStatus = oldBestFriendStatus; bestFriendStatus = oldBestFriendStatus;
// Set the bits // Set the bits
@ -211,13 +185,7 @@ void ChatPacketHandler::HandleFriendRequest(Packet* packet) {
} }
} else { } else {
// Then update the database with this new info. // Then update the database with this new info.
std::unique_ptr<sql::PreparedStatement> updateQuery(Database::CreatePreppedStmt("UPDATE friends SET best_friend = ? WHERE (player_id = ? AND friend_id = ?) OR (player_id = ? AND friend_id = ?) LIMIT 1;")); Database::Get()->SetBestFriendStatus(requestorPlayerID, requestee->playerID, bestFriendStatus);
updateQuery->setUInt(1, bestFriendStatus);
updateQuery->setUInt(2, static_cast<uint32_t>(requestorPlayerID));
updateQuery->setUInt(3, static_cast<uint32_t>(requestee->playerID));
updateQuery->setUInt(4, static_cast<uint32_t>(requestee->playerID));
updateQuery->setUInt(5, static_cast<uint32_t>(requestorPlayerID));
updateQuery->executeUpdate();
// Sent the best friend update here if the value is 3 // Sent the best friend update here if the value is 3
if (bestFriendStatus == 3U) { if (bestFriendStatus == 3U) {
requestee->countOfBestFriends += 1; requestee->countOfBestFriends += 1;
@ -319,11 +287,7 @@ void ChatPacketHandler::HandleFriendResponse(Packet* packet) {
requesteeData.isOnline = true; requesteeData.isOnline = true;
requestor->friends.push_back(requesteeData); requestor->friends.push_back(requesteeData);
std::unique_ptr<sql::PreparedStatement> statement(Database::CreatePreppedStmt("INSERT IGNORE INTO `friends` (`player_id`, `friend_id`, `best_friend`) VALUES (?,?,?);")); Database::Get()->AddFriend(requestor->playerID, requestee->playerID);
statement->setUInt(1, static_cast<uint32_t>(requestor->playerID));
statement->setUInt(2, static_cast<uint32_t>(requestee->playerID));
statement->setInt(3, 0);
statement->execute();
} }
if (serverResponseCode != eAddFriendResponseType::DECLINED) SendFriendResponse(requestor, requestee, serverResponseCode, isAlreadyBestFriends); if (serverResponseCode != eAddFriendResponseType::DECLINED) SendFriendResponse(requestor, requestee, serverResponseCode, isAlreadyBestFriends);
@ -338,25 +302,17 @@ void ChatPacketHandler::HandleRemoveFriend(Packet* packet) {
//we'll have to query the db here to find the user, since you can delete them while they're offline. //we'll have to query the db here to find the user, since you can delete them while they're offline.
//First, we need to find their ID: //First, we need to find their ID:
std::unique_ptr<sql::PreparedStatement> stmt(Database::CreatePreppedStmt("SELECT id FROM charinfo WHERE name=? LIMIT 1;"));
stmt->setString(1, friendName.c_str());
LWOOBJID friendID = 0; LWOOBJID friendID = 0;
std::unique_ptr<sql::ResultSet> res(stmt->executeQuery()); auto friendIdResult = Database::Get()->GetCharacterInfo(friendName);
while (res->next()) { if (friendIdResult) {
friendID = res->getUInt(1); friendID = friendIdResult->id;
} }
// Convert friendID to LWOOBJID // Convert friendID to LWOOBJID
GeneralUtils::SetBit(friendID, eObjectBits::PERSISTENT); GeneralUtils::SetBit(friendID, eObjectBits::PERSISTENT);
GeneralUtils::SetBit(friendID, eObjectBits::CHARACTER); GeneralUtils::SetBit(friendID, eObjectBits::CHARACTER);
std::unique_ptr<sql::PreparedStatement> deletestmt(Database::CreatePreppedStmt("DELETE FROM friends WHERE (player_id = ? AND friend_id = ?) OR (player_id = ? AND friend_id = ?) LIMIT 1;")); Database::Get()->RemoveFriend(playerID, friendID);
deletestmt->setUInt(1, static_cast<uint32_t>(playerID));
deletestmt->setUInt(2, static_cast<uint32_t>(friendID));
deletestmt->setUInt(3, static_cast<uint32_t>(friendID));
deletestmt->setUInt(4, static_cast<uint32_t>(playerID));
deletestmt->execute();
//Now, we need to send an update to notify the sender (and possibly, receiver) that their friendship has been ended: //Now, we need to send an update to notify the sender (and possibly, receiver) that their friendship has been ended:
auto goonA = playerContainer.GetPlayerData(playerID); auto goonA = playerContainer.GetPlayerData(playerID);

View File

@ -78,13 +78,8 @@ int main(int argc, char** argv) {
} }
//Connect to the MySQL Database //Connect to the MySQL Database
std::string mysql_host = Game::config->GetValue("mysql_host");
std::string mysql_database = Game::config->GetValue("mysql_database");
std::string mysql_username = Game::config->GetValue("mysql_username");
std::string mysql_password = Game::config->GetValue("mysql_password");
try { try {
Database::Connect(mysql_host, mysql_database, mysql_username, mysql_password); Database::Connect();
} catch (sql::SQLException& ex) { } catch (sql::SQLException& ex) {
LOG("Got an error while connecting to the database: %s", ex.what()); LOG("Got an error while connecting to the database: %s", ex.what());
Database::Destroy("ChatServer"); Database::Destroy("ChatServer");
@ -96,16 +91,11 @@ int main(int argc, char** argv) {
//Find out the master's IP: //Find out the master's IP:
std::string masterIP; std::string masterIP;
uint32_t masterPort = 1000; uint32_t masterPort = 1000;
sql::PreparedStatement* stmt = Database::CreatePreppedStmt("SELECT ip, port FROM servers WHERE name='master';"); auto masterInfo = Database::Get()->GetMasterInfo();
auto res = stmt->executeQuery(); if (masterInfo) {
while (res->next()) { masterIP = masterInfo->ip;
masterIP = res->getString(1).c_str(); masterPort = masterInfo->port;
masterPort = res->getInt(2);
} }
delete res;
delete stmt;
//It's safe to pass 'localhost' here, as the IP is only used as the external IP. //It's safe to pass 'localhost' here, as the IP is only used as the external IP.
uint32_t maxClients = 50; uint32_t maxClients = 50;
uint32_t ourPort = 1501; uint32_t ourPort = 1501;
@ -158,15 +148,12 @@ int main(int argc, char** argv) {
//Find out the master's IP for absolutely no reason: //Find out the master's IP for absolutely no reason:
std::string masterIP; std::string masterIP;
uint32_t masterPort; uint32_t masterPort;
sql::PreparedStatement* stmt = Database::CreatePreppedStmt("SELECT ip, port FROM servers WHERE name='master';");
auto res = stmt->executeQuery();
while (res->next()) {
masterIP = res->getString(1).c_str();
masterPort = res->getInt(2);
}
delete res; auto masterInfo = Database::Get()->GetMasterInfo();
delete stmt; if (masterInfo) {
masterIP = masterInfo->ip;
masterPort = masterInfo->port;
}
framesSinceLastSQLPing = 0; framesSinceLastSQLPing = 0;
} else framesSinceLastSQLPing++; } else framesSinceLastSQLPing++;

View File

@ -48,14 +48,7 @@ void PlayerContainer::InsertPlayer(Packet* packet) {
m_Players.insert(std::make_pair(data->playerID, data)); m_Players.insert(std::make_pair(data->playerID, data));
LOG("Added user: %s (%llu), zone: %i", data->playerName.c_str(), data->playerID, data->zoneID.GetMapID()); LOG("Added user: %s (%llu), zone: %i", data->playerName.c_str(), data->playerID, data->zoneID.GetMapID());
auto* insertLog = Database::CreatePreppedStmt("INSERT INTO activity_log (character_id, activity, time, map_id) VALUES (?, ?, ?, ?);"); Database::Get()->UpdateActivityLog(data->playerID, eActivityType::PlayerLoggedIn, data->zoneID.GetMapID());
insertLog->setInt(1, data->playerID);
insertLog->setInt(2, 0);
insertLog->setUInt64(3, time(nullptr));
insertLog->setInt(4, data->zoneID.GetMapID());
insertLog->executeUpdate();
} }
void PlayerContainer::RemovePlayer(Packet* packet) { void PlayerContainer::RemovePlayer(Packet* packet) {
@ -92,14 +85,7 @@ void PlayerContainer::RemovePlayer(Packet* packet) {
LOG("Removed user: %llu", playerID); LOG("Removed user: %llu", playerID);
m_Players.erase(playerID); m_Players.erase(playerID);
auto* insertLog = Database::CreatePreppedStmt("INSERT INTO activity_log (character_id, activity, time, map_id) VALUES (?, ?, ?, ?);"); Database::Get()->UpdateActivityLog(playerID, eActivityType::PlayerLoggedOut, player->zoneID.GetMapID());
insertLog->setInt(1, playerID);
insertLog->setInt(2, 1);
insertLog->setUInt64(3, time(nullptr));
insertLog->setInt(4, player->zoneID.GetMapID());
insertLog->executeUpdate();
} }
void PlayerContainer::MuteUpdate(Packet* packet) { void PlayerContainer::MuteUpdate(Packet* packet) {

View File

@ -13,9 +13,8 @@
//! Forward declarations //! Forward declarations
std::unique_ptr<sql::ResultSet> GetModelsFromDatabase();
void WriteSd0Magic(char* input, uint32_t chunkSize); void WriteSd0Magic(char* input, uint32_t chunkSize);
bool CheckSd0Magic(sql::Blob* streamToCheck); bool CheckSd0Magic(std::istream& streamToCheck);
/** /**
* @brief Truncates all models with broken data from the database. * @brief Truncates all models with broken data from the database.
@ -24,28 +23,24 @@ bool CheckSd0Magic(sql::Blob* streamToCheck);
*/ */
uint32_t BrickByBrickFix::TruncateBrokenBrickByBrickXml() { uint32_t BrickByBrickFix::TruncateBrokenBrickByBrickXml() {
uint32_t modelsTruncated{}; uint32_t modelsTruncated{};
auto modelsToTruncate = GetModelsFromDatabase(); auto modelsToTruncate = Database::Get()->GetAllUgcModels();
bool previousCommitValue = Database::GetAutoCommit(); bool previousCommitValue = Database::Get()->GetAutoCommit();
Database::SetAutoCommit(false); Database::Get()->SetAutoCommit(false);
while (modelsToTruncate->next()) { for (auto& model : modelsToTruncate) {
std::unique_ptr<sql::PreparedStatement> ugcModelToDelete(Database::CreatePreppedStmt("DELETE FROM ugc WHERE ugc.id = ?;"));
std::unique_ptr<sql::PreparedStatement> pcModelToDelete(Database::CreatePreppedStmt("DELETE FROM properties_contents WHERE ugc_id = ?;"));
std::string completeUncompressedModel{}; std::string completeUncompressedModel{};
uint32_t chunkCount{}; uint32_t chunkCount{};
uint64_t modelId = modelsToTruncate->getInt(1);
std::unique_ptr<sql::Blob> modelAsSd0(modelsToTruncate->getBlob(2));
// Check that header is sd0 by checking for the sd0 magic. // Check that header is sd0 by checking for the sd0 magic.
if (CheckSd0Magic(modelAsSd0.get())) { if (CheckSd0Magic(model.lxfmlData)) {
while (true) { while (true) {
uint32_t chunkSize{}; uint32_t chunkSize{};
modelAsSd0->read(reinterpret_cast<char*>(&chunkSize), sizeof(uint32_t)); // Extract chunk size from istream model.lxfmlData.read(reinterpret_cast<char*>(&chunkSize), sizeof(uint32_t)); // Extract chunk size from istream
// Check if good here since if at the end of an sd0 file, this will have eof flagged. // Check if good here since if at the end of an sd0 file, this will have eof flagged.
if (!modelAsSd0->good()) break; if (!model.lxfmlData.good()) break;
std::unique_ptr<uint8_t[]> compressedChunk(new uint8_t[chunkSize]); std::unique_ptr<uint8_t[]> compressedChunk(new uint8_t[chunkSize]);
for (uint32_t i = 0; i < chunkSize; i++) { for (uint32_t i = 0; i < chunkSize; i++) {
compressedChunk[i] = modelAsSd0->get(); compressedChunk[i] = model.lxfmlData.get();
} }
// Ignore the valgrind warning about uninitialized values. These are discarded later when we know the actual uncompressed size. // Ignore the valgrind warning about uninitialized values. These are discarded later when we know the actual uncompressed size.
@ -59,7 +54,7 @@ uint32_t BrickByBrickFix::TruncateBrokenBrickByBrickXml() {
completeUncompressedModel.append((char*)uncompressedChunk.get()); completeUncompressedModel.append((char*)uncompressedChunk.get());
completeUncompressedModel.resize(previousSize + actualUncompressedSize); completeUncompressedModel.resize(previousSize + actualUncompressedSize);
} else { } else {
LOG("Failed to inflate chunk %i for model %llu. Error: %i", chunkCount, modelId, err); LOG("Failed to inflate chunk %i for model %llu. Error: %i", chunkCount, model.id, err);
break; break;
} }
chunkCount++; chunkCount++;
@ -75,26 +70,20 @@ uint32_t BrickByBrickFix::TruncateBrokenBrickByBrickXml() {
"</LXFML>", "</LXFML>",
completeUncompressedModel.length() >= 15 ? completeUncompressedModel.length() - 15 : 0) == std::string::npos completeUncompressedModel.length() >= 15 ? completeUncompressedModel.length() - 15 : 0) == std::string::npos
) { ) {
LOG("Brick-by-brick model %llu will be deleted!", modelId); LOG("Brick-by-brick model %llu will be deleted!", model.id);
ugcModelToDelete->setInt64(1, modelsToTruncate->getInt64(1)); Database::Get()->DeleteUgcModelData(model.id);
pcModelToDelete->setInt64(1, modelsToTruncate->getInt64(1));
ugcModelToDelete->execute();
pcModelToDelete->execute();
modelsTruncated++; modelsTruncated++;
} }
} }
} else { } else {
LOG("Brick-by-brick model %llu will be deleted!", modelId); LOG("Brick-by-brick model %llu will be deleted!", model.id);
ugcModelToDelete->setInt64(1, modelsToTruncate->getInt64(1)); Database::Get()->DeleteUgcModelData(model.id);
pcModelToDelete->setInt64(1, modelsToTruncate->getInt64(1));
ugcModelToDelete->execute();
pcModelToDelete->execute();
modelsTruncated++; modelsTruncated++;
} }
} }
Database::Commit(); Database::Get()->Commit();
Database::SetAutoCommit(previousCommitValue); Database::Get()->SetAutoCommit(previousCommitValue);
return modelsTruncated; return modelsTruncated;
} }
@ -106,21 +95,17 @@ uint32_t BrickByBrickFix::TruncateBrokenBrickByBrickXml() {
*/ */
uint32_t BrickByBrickFix::UpdateBrickByBrickModelsToSd0() { uint32_t BrickByBrickFix::UpdateBrickByBrickModelsToSd0() {
uint32_t updatedModels = 0; uint32_t updatedModels = 0;
auto modelsToUpdate = GetModelsFromDatabase(); auto modelsToUpdate = Database::Get()->GetAllUgcModels();
auto previousAutoCommitState = Database::GetAutoCommit(); auto previousAutoCommitState = Database::Get()->GetAutoCommit();
Database::SetAutoCommit(false); Database::Get()->SetAutoCommit(false);
std::unique_ptr<sql::PreparedStatement> insertionStatement(Database::CreatePreppedStmt("UPDATE ugc SET lxfml = ? WHERE id = ?;")); for (auto& model : modelsToUpdate) {
while (modelsToUpdate->next()) {
int64_t modelId = modelsToUpdate->getInt64(1);
std::unique_ptr<sql::Blob> oldLxfml(modelsToUpdate->getBlob(2));
// Check if the stored blob starts with zlib magic (0x78 0xDA - best compression of zlib) // Check if the stored blob starts with zlib magic (0x78 0xDA - best compression of zlib)
// If it does, convert it to sd0. // If it does, convert it to sd0.
if (oldLxfml->get() == 0x78 && oldLxfml->get() == 0xDA) { if (model.lxfmlData.get() == 0x78 && model.lxfmlData.get() == 0xDA) {
// Get and save size of zlib compressed chunk. // Get and save size of zlib compressed chunk.
oldLxfml->seekg(0, std::ios::end); model.lxfmlData.seekg(0, std::ios::end);
uint32_t oldLxfmlSize = static_cast<uint32_t>(oldLxfml->tellg()); uint32_t oldLxfmlSize = static_cast<uint32_t>(model.lxfmlData.tellg());
oldLxfml->seekg(0); model.lxfmlData.seekg(0);
// Allocate 9 extra bytes. 5 for sd0 magic, 4 for the only zlib compressed size. // Allocate 9 extra bytes. 5 for sd0 magic, 4 for the only zlib compressed size.
uint32_t oldLxfmlSizeWithHeader = oldLxfmlSize + 9; uint32_t oldLxfmlSizeWithHeader = oldLxfmlSize + 9;
@ -128,34 +113,27 @@ uint32_t BrickByBrickFix::UpdateBrickByBrickModelsToSd0() {
WriteSd0Magic(sd0ConvertedModel.get(), oldLxfmlSize); WriteSd0Magic(sd0ConvertedModel.get(), oldLxfmlSize);
for (uint32_t i = 9; i < oldLxfmlSizeWithHeader; i++) { for (uint32_t i = 9; i < oldLxfmlSizeWithHeader; i++) {
sd0ConvertedModel.get()[i] = oldLxfml->get(); sd0ConvertedModel.get()[i] = model.lxfmlData.get();
} }
std::string outputString(sd0ConvertedModel.get(), oldLxfmlSizeWithHeader); std::string outputString(sd0ConvertedModel.get(), oldLxfmlSizeWithHeader);
std::istringstream outputStringStream(outputString); std::istringstream outputStringStream(outputString);
insertionStatement->setBlob(1, static_cast<std::istream*>(&outputStringStream));
insertionStatement->setInt64(2, modelId);
try { try {
insertionStatement->executeUpdate(); Database::Get()->UpdateUgcModelData(model.id, outputStringStream);
LOG("Updated model %i to sd0", modelId); LOG("Updated model %i to sd0", model.id);
updatedModels++; updatedModels++;
} catch (sql::SQLException exception) { } catch (sql::SQLException exception) {
LOG("Failed to update model %i. This model should be inspected manually to see why." LOG("Failed to update model %i. This model should be inspected manually to see why."
"The database error is %s", modelId, exception.what()); "The database error is %s", model.id, exception.what());
} }
} }
} }
Database::Commit(); Database::Get()->Commit();
Database::SetAutoCommit(previousAutoCommitState); Database::Get()->SetAutoCommit(previousAutoCommitState);
return updatedModels; return updatedModels;
} }
std::unique_ptr<sql::ResultSet> GetModelsFromDatabase() {
std::unique_ptr<sql::PreparedStatement> modelsRawDataQuery(Database::CreatePreppedStmt("SELECT id, lxfml FROM ugc;"));
return std::unique_ptr<sql::ResultSet>(modelsRawDataQuery->executeQuery());
}
/** /**
* @brief Writes sd0 magic at the front of a char* * @brief Writes sd0 magic at the front of a char*
* *
@ -171,6 +149,6 @@ void WriteSd0Magic(char* input, uint32_t chunkSize) {
*reinterpret_cast<uint32_t*>(input + 5) = chunkSize; // Write the integer to the character array *reinterpret_cast<uint32_t*>(input + 5) = chunkSize; // Write the integer to the character array
} }
bool CheckSd0Magic(sql::Blob* streamToCheck) { bool CheckSd0Magic(std::istream& streamToCheck) {
return streamToCheck->get() == 's' && streamToCheck->get() == 'd' && streamToCheck->get() == '0' && streamToCheck->get() == 0x01 && streamToCheck->get() == 0xFF; return streamToCheck.get() == 's' && streamToCheck.get() == 'd' && streamToCheck.get() == '0' && streamToCheck.get() == 0x01 && streamToCheck.get() == 0xFF;
} }

View File

@ -3,6 +3,8 @@
// Custom Classes // Custom Classes
#include "CDTable.h" #include "CDTable.h"
#include <unordered_map>
enum class eReplicaComponentType : uint32_t; enum class eReplicaComponentType : uint32_t;
struct CDComponentsRegistry { struct CDComponentsRegistry {
unsigned int id; //!< The LOT is used as the ID unsigned int id; //!< The LOT is used as the ID

View File

@ -1,4 +1,4 @@
set(DDATABASE_TABLES_SOURCES "CDActivitiesTable.cpp" set(DDATABASE_CDCLIENTDATABASE_CDCLIENTTABLES_SOURCES "CDActivitiesTable.cpp"
"CDActivityRewardsTable.cpp" "CDActivityRewardsTable.cpp"
"CDAnimationsTable.cpp" "CDAnimationsTable.cpp"
"CDBehaviorParameterTable.cpp" "CDBehaviorParameterTable.cpp"

View File

@ -0,0 +1,12 @@
set(DDATABASE_CDCLIENTDATABASE_SOURCES
"CDClientDatabase.cpp"
"CDClientManager.cpp"
)
add_subdirectory(CDClientTables)
foreach(file ${DDATABASE_CDCLIENTDATABASE_CDCLIENTTABLES_SOURCES})
set(DDATABASE_CDCLIENTDATABASE_SOURCES ${DDATABASE_CDCLIENTDATABASE_SOURCES} "CDClientTables/${file}")
endforeach()
set(DDATABASE_CDCLIENTDATABASE_SOURCES ${DDATABASE_CDCLIENTDATABASE_SOURCES} PARENT_SCOPE)

View File

@ -1,12 +1,15 @@
set(DDATABASE_SOURCES "CDClientDatabase.cpp" set(DDATABASE_SOURCES)
"CDClientManager.cpp"
"Database.cpp"
"MigrationRunner.cpp")
add_subdirectory(Tables) add_subdirectory(CDClientDatabase)
foreach(file ${DDATABASE_TABLES_SOURCES}) foreach(file ${DDATABASE_CDCLIENTDATABASE_SOURCES})
set(DDATABASE_SOURCES ${DDATABASE_SOURCES} "Tables/${file}") set(DDATABASE_SOURCES ${DDATABASE_SOURCES} "CDClientDatabase/${file}")
endforeach()
add_subdirectory(GameDatabase)
foreach(file ${DDATABASE_GAMEDATABASE_SOURCES})
set(DDATABASE_SOURCES ${DDATABASE_SOURCES} "GameDatabase/${file}")
endforeach() endforeach()
add_library(dDatabase STATIC ${DDATABASE_SOURCES}) add_library(dDatabase STATIC ${DDATABASE_SOURCES})

View File

@ -1,118 +0,0 @@
#include "Database.h"
#include "Game.h"
#include "dConfig.h"
#include "Logger.h"
using namespace std;
#pragma warning (disable:4251) //Disables SQL warnings
sql::Driver* Database::driver;
sql::Connection* Database::con;
sql::Properties Database::props;
std::string Database::database;
void Database::Connect(const string& host, const string& database, const string& username, const string& password) {
//To bypass debug issues:
const char* szDatabase = database.c_str();
const char* szUsername = username.c_str();
const char* szPassword = password.c_str();
driver = sql::mariadb::get_driver_instance();
sql::Properties properties;
// The mariadb connector is *supposed* to handle unix:// and pipe:// prefixes to hostName, but there are bugs where
// 1) it tries to parse a database from the connection string (like in tcp://localhost:3001/darkflame) based on the
// presence of a /
// 2) even avoiding that, the connector still assumes you're connecting with a tcp socket
// So, what we do in the presence of a unix socket or pipe is to set the hostname to the protocol and localhost,
// which avoids parsing errors while still ensuring the correct connection type is used, and then setting the appropriate
// property manually (which the URL parsing fails to do)
const std::string UNIX_PROTO = "unix://";
const std::string PIPE_PROTO = "pipe://";
if (host.find(UNIX_PROTO) == 0) {
properties["hostName"] = "unix://localhost";
properties["localSocket"] = host.substr(UNIX_PROTO.length()).c_str();
} else if (host.find(PIPE_PROTO) == 0) {
properties["hostName"] = "pipe://localhost";
properties["pipe"] = host.substr(PIPE_PROTO.length()).c_str();
} else {
properties["hostName"] = host.c_str();
}
properties["user"] = szUsername;
properties["password"] = szPassword;
properties["autoReconnect"] = "true";
Database::props = properties;
Database::database = database;
Database::Connect();
}
void Database::Connect() {
// `connect(const Properties& props)` segfaults in windows debug, but
// `connect(const SQLString& host, const SQLString& user, const SQLString& pwd)` doesn't handle pipes/unix sockets correctly
if (Database::props.find("localSocket") != Database::props.end() || Database::props.find("pipe") != Database::props.end()) {
con = driver->connect(Database::props);
} else {
con = driver->connect(Database::props["hostName"].c_str(), Database::props["user"].c_str(), Database::props["password"].c_str());
}
con->setSchema(Database::database.c_str());
}
void Database::Destroy(std::string source, bool log) {
if (!con) return;
if (log) {
if (source != "") LOG("Destroying MySQL connection from %s!", source.c_str());
else LOG("Destroying MySQL connection!");
}
con->close();
delete con;
} //Destroy
sql::Statement* Database::CreateStmt() {
sql::Statement* toReturn = con->createStatement();
return toReturn;
} //CreateStmt
sql::PreparedStatement* Database::CreatePreppedStmt(const std::string& query) {
const char* test = query.c_str();
size_t size = query.length();
sql::SQLString str(test, size);
if (!con) {
Connect();
LOG("Trying to reconnect to MySQL");
}
if (!con->isValid() || con->isClosed()) {
delete con;
con = nullptr;
Connect();
LOG("Trying to reconnect to MySQL from invalid or closed connection");
}
auto* stmt = con->prepareStatement(str);
return stmt;
} //CreatePreppedStmt
void Database::Commit() {
Database::con->commit();
}
bool Database::GetAutoCommit() {
// TODO This should not just access a pointer. A future PR should update this
// to check for null and throw an error if the connection is not valid.
return con->getAutoCommit();
}
void Database::SetAutoCommit(bool value) {
// TODO This should not just access a pointer. A future PR should update this
// to check for null and throw an error if the connection is not valid.
Database::con->setAutoCommit(value);
}

View File

@ -1,31 +0,0 @@
#pragma once
#include <string>
#include <conncpp.hpp>
class MySqlException : public std::runtime_error {
public:
MySqlException() : std::runtime_error("MySQL error!") {}
MySqlException(const std::string& msg) : std::runtime_error(msg.c_str()) {}
};
class Database {
private:
static sql::Driver* driver;
static sql::Connection* con;
static sql::Properties props;
static std::string database;
public:
static void Connect(const std::string& host, const std::string& database, const std::string& username, const std::string& password);
static void Connect();
static void Destroy(std::string source = "", bool log = true);
static sql::Statement* CreateStmt();
static sql::PreparedStatement* CreatePreppedStmt(const std::string& query);
static void Commit();
static bool GetAutoCommit();
static void SetAutoCommit(bool value);
static std::string GetDatabase() { return database; }
static sql::Properties GetProperties() { return props; }
};

View File

@ -0,0 +1,12 @@
set(DDATABASE_GAMEDATABASE_SOURCES
"Database.cpp"
"MigrationRunner.cpp"
)
add_subdirectory(MySQL)
foreach(file ${DDATABSE_DATABSES_MYSQL_SOURCES})
set(DDATABASE_GAMEDATABASE_SOURCES ${DDATABASE_GAMEDATABASE_SOURCES} "MySQL/${file}")
endforeach()
set(DDATABASE_GAMEDATABASE_SOURCES ${DDATABASE_GAMEDATABASE_SOURCES} PARENT_SCOPE)

View File

@ -0,0 +1,40 @@
#include "Database.h"
#include "Game.h"
#include "dConfig.h"
#include "Logger.h"
#include "MySQLDatabase.h"
#include "DluAssert.h"
#pragma warning (disable:4251) //Disables SQL warnings
namespace {
GameDatabase* database = nullptr;
}
void Database::Connect() {
if (database) {
LOG("Tried to connect to database when it's already connected!");
return;
}
database = new MySQLDatabase();
database->Connect();
}
GameDatabase* Database::Get() {
if (!database) {
LOG("Tried to get database when it's not connected!");
Connect();
}
return database;
}
void Database::Destroy(std::string source) {
if (database) {
database->Destroy(source);
delete database;
database = nullptr;
} else {
LOG("Trying to destroy database when it's not connected!");
}
}

View File

@ -0,0 +1,12 @@
#pragma once
#include <string>
#include <conncpp.hpp>
#include "GameDatabase.h"
namespace Database {
void Connect();
GameDatabase* Get();
void Destroy(std::string source = "");
};

View File

@ -0,0 +1,55 @@
#ifndef __GAMEDATABASE__H__
#define __GAMEDATABASE__H__
#include <optional>
#include "ILeaderboard.h"
#include "IPlayerCheatDetections.h"
#include "ICommandLog.h"
#include "IMail.h"
#include "IObjectIdTracker.h"
#include "IPlayKeys.h"
#include "IServers.h"
#include "IBugReports.h"
#include "IPropertyContents.h"
#include "IProperty.h"
#include "IPetNames.h"
#include "ICharXml.h"
#include "IMigrationHistory.h"
#include "IUgc.h"
#include "IFriends.h"
#include "ICharInfo.h"
#include "IAccounts.h"
#include "IActivityLog.h"
namespace sql {
class Statement;
class PreparedStatement;
};
#ifdef _DEBUG
# define DLU_SQL_TRY_CATCH_RETHROW(x) do { try { x; } catch (sql::SQLException& ex) { LOG("SQL Error: %s", ex.what()); throw; } } while(0)
#else
# define DLU_SQL_TRY_CATCH_RETHROW(x) x
#endif // _DEBUG
class GameDatabase :
public IPlayKeys, public ILeaderboard, public IObjectIdTracker, public IServers,
public IMail, public ICommandLog, public IPlayerCheatDetections, public IBugReports,
public IPropertyContents, public IProperty, public IPetNames, public ICharXml,
public IMigrationHistory, public IUgc, public IFriends, public ICharInfo,
public IAccounts, public IActivityLog {
public:
virtual ~GameDatabase() = default;
// TODO: These should be made private.
virtual void Connect() = 0;
virtual void Destroy(std::string source = "") = 0;
virtual void ExecuteCustomQuery(const std::string_view query) = 0;
virtual sql::PreparedStatement* CreatePreppedStmt(const std::string& query) = 0;
virtual void Commit() = 0;
virtual bool GetAutoCommit() = 0;
virtual void SetAutoCommit(bool value) = 0;
virtual void DeleteCharacter(const uint32_t characterId) = 0;
};
#endif //!__GAMEDATABASE__H__

View File

@ -0,0 +1,37 @@
#ifndef __IACCOUNTS__H__
#define __IACCOUNTS__H__
#include <cstdint>
#include <optional>
#include <string_view>
enum class eGameMasterLevel : uint8_t;
class IAccounts {
public:
struct Info {
std::string bcryptPassword;
uint32_t id{};
uint32_t playKeyId{};
bool banned{};
bool locked{};
eGameMasterLevel maxGmLevel{};
};
// Get the account info for the given username.
virtual std::optional<IAccounts::Info> GetAccountInfo(const std::string_view username) = 0;
// Update the account's unmute time.
virtual void UpdateAccountUnmuteTime(const uint32_t accountId, const uint64_t timeToUnmute) = 0;
// Update the account's ban status.
virtual void UpdateAccountBan(const uint32_t accountId, const bool banned) = 0;
// Update the account's password.
virtual void UpdateAccountPassword(const uint32_t accountId, const std::string_view bcryptpassword) = 0;
// Add a new account to the database.
virtual void InsertNewAccount(const std::string_view username, const std::string_view bcryptpassword) = 0;
};
#endif //!__IACCOUNTS__H__

View File

@ -0,0 +1,19 @@
#ifndef __IACTIVITYLOG__H__
#define __IACTIVITYLOG__H__
#include <cstdint>
#include "dCommonVars.h"
enum class eActivityType : uint32_t {
PlayerLoggedIn,
PlayerLoggedOut,
};
class IActivityLog {
public:
// Update the activity log for the given account.
virtual void UpdateActivityLog(const uint32_t characterId, const eActivityType activityType, const LWOMAPID mapId) = 0;
};
#endif //!__IACTIVITYLOG__H__

View File

@ -0,0 +1,20 @@
#ifndef __IBUGREPORTS__H__
#define __IBUGREPORTS__H__
#include <cstdint>
#include <string_view>
class IBugReports {
public:
struct Info {
std::string body;
std::string clientVersion;
std::string otherPlayer;
std::string selection;
uint32_t characterId{};
};
// Add a new bug report to the database.
virtual void InsertNewBugReport(const Info& info) = 0;
};
#endif //!__IBUGREPORTS__H__

Some files were not shown because too many files have changed in this diff Show More