Files
DarkflameServer/dGame/dUtilities/UgcManifest.cpp
Aaron Kimbrell 9f29f65686 feat(ugc): answer the client's UGC manifest requests without 3D services
With UGCUSE3DSERVICES=7:0 (the client's default) the client asks its world for
a blueprint file's MD5 and size (REQUEST_UGC_MANIFEST_INFO, world 27) and then
downloads BrickModels/UserMade/<id % 1000>/<id>.<ext>.sd0. Layouts checked in
the 1.10.64 client: the request is a u64 blueprint id and a u8 resource type;
the answer (UGC_MANIFEST_RESPONSE, client 60) repeats them and adds a u8 valid,
the u32 size and the 16 byte MD5 of the inflated file, 37 bytes after the 0x53
exactly or the client drops it.

- dNet: WorldPackets::RequestUgcManifestInfo, ClientPackets::UgcManifestResponse
  (eUgcResourceType), with byte tests against the client's layouts.
- Database: ugc_file_checksums (per model or module combination and file) and
  ugc_modular_build.combination_id (migrations 89 / 72), GetUgcFileChecksum
  looks a blueprint up as a model, else as a build through its combination.
- UGC server: every download is also written as .sd0 (Sd0::Compress); workers
  hand the checksums back and the main thread stores them; old items get their
  sd0 icon and checksum, and builds their combination id, once at start-up, a
  few per tick; serves <dir>/BrickModels/UserMade/<bucket>/<id>.<ext>.sd0 under
  client_path, /<folder>/UserBrickModels and the root (.hkx 404).
- World: UgcManifest answers on the main thread with one indexed query per
  request; files not made yet are answered when they are (looked at again
  every 5 seconds), and a model in its quiet period is made right away. No
  worker threads, HTTP or file reads in the world.

Off by default (ugc_manifest=0): checked in game, the client then downloads
from http://127.0.0.1:80/lwoclient/UserBrickModels/ whatever its boot.cfg says
and logs the player out when it can't connect, so icons need the UGC server on
port 80 of each player's machine. docs/UgcServer.md has the details.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 17:47:38 -05:00

103 lines
3.9 KiB
C++

#include "UgcManifest.h"
#include <chrono>
#include <optional>
#include <string_view>
#include <vector>
#include "ClientPackets.h"
#include "Database.h"
#include "dConfig.h"
#include "Game.h"
#include "Logger.h"
namespace {
// How often the waiting requests are looked up again, how long they wait at most and how many there can be
constexpr auto RETRY_INTERVAL = std::chrono::seconds(5);
constexpr auto MAX_WAIT = std::chrono::minutes(15);
constexpr size_t MAX_WAITING = 512;
struct Waiting {
SystemAddress sysAddr;
LWOOBJID blueprintId{};
eUgcResourceType resourceType{};
std::chrono::steady_clock::time_point since;
};
std::vector<Waiting> g_Waiting;
std::chrono::steady_clock::time_point g_NextRetry{};
// The UGC server's name for a blueprint's file of this type; nothing for the ones it doesn't make
std::optional<std::string_view> FileName(eUgcResourceType type) {
switch (type) {
case eUgcResourceType::DDS: return "icon.dds";
case eUgcResourceType::NIF: return "model.nif";
default: return std::nullopt;
}
}
bool Enabled() {
// Off unless set: the 1.10.64 client downloads from http://127.0.0.1:80 whatever its boot.cfg says, and one that
// can't connect there is logged out (docs/UgcServer.md)
return Game::config->GetValue("ugc_manifest") == "1";
}
int HexDigit(char c) {
if (c >= '0' && c <= '9') return c - '0';
if (c >= 'a' && c <= 'f') return c - 'a' + 10;
if (c >= 'A' && c <= 'F') return c - 'A' + 10;
return -1;
}
// Answers when the file's checksum is known; false when it isn't (yet)
bool TryAnswer(const SystemAddress& sysAddr, LWOOBJID blueprintId, eUgcResourceType resourceType) {
const auto file = FileName(resourceType);
if (!file) return false;
const auto checksum = Database::Get()->GetUgcFileChecksum(blueprintId, *file);
if (!checksum || checksum->md5.size() != 32) return false;
ClientPackets::UgcManifestResponse response;
response.blueprintId = blueprintId;
response.resourceType = resourceType;
response.fileSize = checksum->size;
for (size_t i = 0; i < response.md5.size(); i++) {
const auto high = HexDigit(checksum->md5[i * 2]);
const auto low = HexDigit(checksum->md5[i * 2 + 1]);
if (high < 0 || low < 0) return false;
response.md5[i] = static_cast<uint8_t>(high << 4 | low);
}
response.valid = true;
response.Send(sysAddr);
LOG_DEBUG("Sent the UGC manifest of %llu (type %i): %s, %u bytes", static_cast<unsigned long long>(blueprintId),
static_cast<int>(resourceType), checksum->md5.c_str(), checksum->size);
return true;
}
}
void UgcManifest::OnRequest(const SystemAddress& sysAddr, LWOOBJID blueprintId, eUgcResourceType resourceType) {
if (sysAddr == UNASSIGNED_SYSTEM_ADDRESS || !Enabled() || !FileName(resourceType)) return;
if (TryAnswer(sysAddr, blueprintId, resourceType)) return;
// Not made yet: a client wants it, so a model still in its quiet period after a save is made now (as when a client
// asks the UGC server for it), and the request is answered when it's made
Database::Get()->ExpediteUgcModel(blueprintId);
for (const auto& waiting : g_Waiting) {
if (waiting.sysAddr == sysAddr && waiting.blueprintId == blueprintId && waiting.resourceType == resourceType) return;
}
if (g_Waiting.size() >= MAX_WAITING) g_Waiting.erase(g_Waiting.begin());
g_Waiting.push_back({ sysAddr, blueprintId, resourceType, std::chrono::steady_clock::now() });
}
void UgcManifest::Update() {
if (g_Waiting.empty()) return;
const auto now = std::chrono::steady_clock::now();
if (now < g_NextRetry) return;
g_NextRetry = now + RETRY_INTERVAL;
std::erase_if(g_Waiting, [now](const Waiting& waiting) {
return now - waiting.since > MAX_WAIT || TryAnswer(waiting.sysAddr, waiting.blueprintId, waiting.resourceType);
});
}
void UgcManifest::OnDisconnect(const SystemAddress& sysAddr) {
std::erase_if(g_Waiting, [&sysAddr](const Waiting& waiting) { return waiting.sysAddr == sysAddr; });
}