feat(ugc): answer the client's UGC manifest requests without 3D services

With UGCUSE3DSERVICES=7:0 (the client's default) the client asks its world for
a blueprint file's MD5 and size (REQUEST_UGC_MANIFEST_INFO, world 27) and then
downloads BrickModels/UserMade/<id % 1000>/<id>.<ext>.sd0. Layouts checked in
the 1.10.64 client: the request is a u64 blueprint id and a u8 resource type;
the answer (UGC_MANIFEST_RESPONSE, client 60) repeats them and adds a u8 valid,
the u32 size and the 16 byte MD5 of the inflated file, 37 bytes after the 0x53
exactly or the client drops it.

- dNet: WorldPackets::RequestUgcManifestInfo, ClientPackets::UgcManifestResponse
  (eUgcResourceType), with byte tests against the client's layouts.
- Database: ugc_file_checksums (per model or module combination and file) and
  ugc_modular_build.combination_id (migrations 89 / 72), GetUgcFileChecksum
  looks a blueprint up as a model, else as a build through its combination.
- UGC server: every download is also written as .sd0 (Sd0::Compress); workers
  hand the checksums back and the main thread stores them; old items get their
  sd0 icon and checksum, and builds their combination id, once at start-up, a
  few per tick; serves <dir>/BrickModels/UserMade/<bucket>/<id>.<ext>.sd0 under
  client_path, /<folder>/UserBrickModels and the root (.hkx 404).
- World: UgcManifest answers on the main thread with one indexed query per
  request; files not made yet are answered when they are (looked at again
  every 5 seconds), and a model in its quiet period is made right away. No
  worker threads, HTTP or file reads in the world.

Off by default (ugc_manifest=0): checked in game, the client then downloads
from http://127.0.0.1:80/lwoclient/UserBrickModels/ whatever its boot.cfg says
and logs the player out when it can't connect, so icons need the UGC server on
port 80 of each player's machine. docs/UgcServer.md has the details.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Aaron Kimbrell
2026-09-27 17:46:47 -05:00
parent ceec638e52
commit 79235702b3
33 changed files with 706 additions and 14 deletions

View File

@@ -3,6 +3,7 @@
#include <cstring>
#include <filesystem>
#include <fstream>
#include <sstream>
#include <unistd.h>
#include <glm/gtc/matrix_transform.hpp>
@@ -22,6 +23,7 @@
#include "UgcRender.h"
#include "UgcStorage.h"
#include "UgcThrottle.h"
#include "Sd0.h"
#include "ZCompression.h"
#include "json.hpp"
@@ -269,6 +271,28 @@ TEST(UgcFormats, ImagesAndChecksums) {
EXPECT_EQ(dds[128 + 2], 10); // stored BGRA
EXPECT_EQ(UgcFormats::Md5Hex("abc"), "900150983cd24fb0d6963f7d28e17f72");
EXPECT_NE(UgcFormats::ChecksumXml("abc").find("<Checksum><MD5>900150983cd24fb0d6963f7d28e17f72</MD5><Filesize>3</Filesize></Checksum>"), std::string::npos);
std::string md5;
uint32_t size{};
ASSERT_TRUE(UgcFormats::ReadChecksumXml(UgcFormats::ChecksumXml("abc"), md5, size));
EXPECT_EQ(md5, "900150983cd24fb0d6963f7d28e17f72");
EXPECT_EQ(size, 3u);
EXPECT_FALSE(UgcFormats::ReadChecksumXml("<Checksum><MD5>abc</MD5><Filesize>3</Filesize></Checksum>", md5, size));
EXPECT_FALSE(UgcFormats::ReadChecksumXml("<Checksum><MD5>900150983cd24fb0d6963f7d28e17f72</MD5><Filesize>x</Filesize></Checksum>", md5, size));
}
// A download is written for both of the client's modes: .gz and .checksum (3D services) and .sd0 (without), all
// holding the same file
TEST(UgcJobs, AddsTheDownloadForBothClientModes) {
UgcStorage::Files files;
UgcJobs::AddDownload(files, "icon.dds", "abc");
EXPECT_EQ(ZCompression::Gunzip(files.at("icon.dds.gz")).value_or(""), "abc");
std::istringstream sd0(files.at("icon.dds.sd0"));
EXPECT_EQ(Sd0(sd0).GetAsStringUncompressed(), "abc");
std::string md5;
uint32_t size{};
ASSERT_TRUE(UgcFormats::ReadChecksumXml(files.at("icon.dds.checksum"), md5, size));
EXPECT_EQ(md5, "900150983cd24fb0d6963f7d28e17f72");
EXPECT_EQ(size, 3u);
}
TEST(UgcModular, ParsesTheCdClientData) {