Mirrored_Repos/DarkflameServer
1
0
Fork 0
mirror of https://github.com/DarkflameUniverse/DarkflameServer.git synced 2026-10-17 10:12:50 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(ugc): LU Toolbox worker in a headless Blender

Browse Source 
UgcToolbox runs LU Toolbox itself in an external Blender that stays up
and makes one model after another (dlu_toolbox_worker.py runs
LU-Toolbox-Standalone's steps: import, Process Model, Bake Lighting,
niftools export). JSON lines over its stdin and original stdout; it is
restarted after a crash, a timeout, 100 models or new settings, runs at
worker_nice with toolbox_threads, and its CPU time is charged to the
worker, which pauses it (SIGSTOP) while the CPU budget is overdrawn.
ProcessModelToolbox reads LU Toolbox's .nif back, counts its triangles
and draws the icon from it. Tests: option parsing, the fallback, the
protocol framing, the worker with a stand-in Blender (crash, hang,
failure, give-up), and LU Toolbox itself when DLU_TEST_TOOLBOX_* are set.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Aaron Kimbrell
2026-09-29 16:28:30 -05:00
parent eb201d7b86
commit 6ae6eac64e
13 changed files with 1585 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
dUgcServer/Processing/UgcJobs.cpp
View File

@@ -179,6 +179,7 @@ namespace UgcJobs {
void ApplyOptions(Settings& settings, const UgcProcessOptions::Choice& choice) {
if (const auto rays = UgcRays::Parse(choice.rays)) settings.ao.rays = settings.icon.ao.rays = *rays;
if (const auto denoise = UgcRender::ParseDenoise(choice.denoise)) settings.icon.denoise = *denoise;
if (UgcProcessOptions::Contains(UgcProcessOptions::PROCESSOR, choice.processor)) settings.processor = choice.processor;
}
UgcProcessOptions::Choice MadeWith(const Settings& settings) {

18
dUgcServer/Processing/UgcJobs.h
View File

@@ -16,6 +16,10 @@
#include "UgcRender.h"
#include "UgcStorage.h"
namespace UgcToolbox {
class Worker;
}
/**
* The work a UGC worker thread does for one item: everything from the stored LXFML (or a modular build's modules) to
* the finished files. No database, network or CDClient: the main thread gathers the input and stores the result.
@@ -63,6 +67,9 @@ namespace UgcJobs {
UgcRender::AoOptions ao; // Bake Lighting (AO Only)
UgcRender::IconOptions icon; // from the icon_* settings (UgcIconParams); presets and overrides go over it
uint32_t maxBricks{}; // a model with more fails; 0: no limit
// processor: what makes a player model's files, UgcProcessOptions::PROCESSOR (native: ProcessModel;
// toolbox-blender: ProcessModelToolbox, when UgcToolbox can run, else native)
std::string processor{ UgcProcessOptions::NATIVE };
};
/**
@@ -98,6 +105,17 @@ namespace UgcJobs {
Outcome ProcessModel(const std::string& blob, UgcBricks::BrickLibrary& library, const Settings& settings, uint64_t seed = 0,
const UgcIconParams::Values& iconValues = {});
/**
* A player model made by LU Toolbox itself in Blender (processor=toolbox-blender, docs/UgcServer.md): its .nif is
* LU Toolbox's (import, Process Model, Bake Lighting, niftools export), checked by reading it back; the icon is
* drawn from it by the icon renderer, as a native model's is from its .nif (not denoised: there is no .nif before
* the bake). stats.json has the bricks and triangles before (the bricks' own meshes) and after (the .nif's), and
* LU Toolbox's steps' times. Blender's CPU time is charged to the calling thread (UgcThrottle::Charge), which keeps
* the CPU budget by pausing Blender. `id` names the worker's files.
*/
Outcome ProcessModelToolbox(const std::string& blob, UgcBricks::BrickLibrary& library, const Settings& settings, UgcToolbox::Worker& toolbox,
uint64_t id, const UgcIconParams::Values& iconValues = {});
// A model's icon files (icon.png, icon.dds download) drawn from its .nif (LOD 0), its metal and glow groups by
// `tagLooks` (Shaders::TagLooks); false (and `error`) when the .nif can't be read
// (the groups drawn over others, `overlayTags` (Shaders::OverlayTags), left out). `plainNif`: its model.noao.nif, the

11
dUgcServer/Processing/UgcThrottle.cpp
View File

@@ -75,6 +75,10 @@ namespace UgcThrottle {
bool IsCancelled() { return g_Cancel; }
void Checkpoint() {
Checkpoint({});
}
void Checkpoint(const std::function<void(bool)>& pausing) {
if (g_Cancel) throw Cancelled{};
const double budget = g_Budget;
if (budget <= 0.0) return;
@@ -99,10 +103,15 @@ namespace UgcThrottle {
g_LastSleep = UnixMs();
// In short sleeps, so a cancel isn't held up by a long wait
const auto until = std::chrono::steady_clock::now() + std::chrono::duration_cast<std::chrono::steady_clock::duration>(std::chrono::duration<double>(wait));
if (pausing) pausing(true);
while (std::chrono::steady_clock::now() < until) {
if (g_Cancel) throw Cancelled{};
if (g_Cancel) {
if (pausing) pausing(false);
throw Cancelled{};
}
std::this_thread::sleep_for(std::min<std::chrono::steady_clock::duration>(until - std::chrono::steady_clock::now(), std::chrono::milliseconds(100)));
}
if (pausing) pausing(false);
// Time asleep costs no CPU; don't count this call's own bookkeeping twice
t_LastCpu = ThreadCpuSeconds();
}

5
dUgcServer/Processing/UgcThrottle.h
View File

@@ -1,6 +1,7 @@
#pragma once
#include <cstdint>
#include <functional>
#include <string>
/**
@@ -18,6 +19,10 @@ namespace UgcThrottle {
// called, so a job stops in moments when the server shuts down.
void Checkpoint();
// The same, for work done by another process for this thread (LU Toolbox's Blender, charged with Charge first):
// `pausing` is called with true before the sleep and false after it, to stop that process meanwhile
void Checkpoint(const std::function<void(bool)>& pausing);
// Thrown by Checkpoint while cancelled: the job is abandoned, not failed (its row stays waiting)
struct Cancelled {};
// Stop (true) or allow (false) every job's work: set when the server stops, cleared when it starts