Mirrored_Repos/DarkflameServer
1
0
Fork 0
mirror of https://github.com/DarkflameUniverse/DarkflameServer.git synced 2026-10-02 10:53:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(dashboard): scenery models keep their colors, lit like the game

Browse Source 
Glom models (multishader) drew their trees, rocks, fences and water white.
Their conversions kept on disk from before meshes carried their multishader
tag were still served (the cache format version was never bumped), so every
part fell back to the LEGO shader, whose texture alpha lays the (mostly grey)
texture over the vertex colors that hold the actual colors. Browsers also
kept those models for a week.

- Bump the conversion format so old conversions are made again, and put it
  in the manifests; the viewers add it to model and texture URLs.
- Light scenery as the client's shaders do: the scene's sun and ambient
  light from its .lvl (read as level_read_lighting_info, 0x0102f8f0, and
  EnvironmentManager::SetLightEnv, 0x01088aa0, do), per vertex, clamped,
  instead of the view's own lights, environment map and tone mapping. The
  zone takes the lighting most of its objects' scenes have.
- Programmable shaders read vertex colors and ignore NiMaterialProperty's
  color and alpha; unlit and untextured shaders (by mapShaders gameValue)
  leave out lighting or the texture. Fixed function stays as it was.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Aaron Kimbrell
2026-09-27 21:38:43 -05:00
parent f252c6ff26
commit c0054d8fa3
9 changed files with 456 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files

33
dCommon/NifFile.cpp
View File

@@ -661,6 +661,39 @@ namespace NifFile {
}
}
uint8_t ShaderLookFor(int32_t shader) {
// By the techniques of each mapShaders row (its label names them: "NL" no lighting, "NT" no texture, "VC"
// vertex colors), e.g. 38 "Basic VC" is Technique_Basic_Lighting_VertColor, 33 "Basic NL VC NT"
// Technique_Basic_NoLighting_VertColor_NoTexture, 32 "Basic NL Material" Technique_Basic_Material_NoLighting
switch (shader) {
// Basic NL Material, Over Everything Material Unlit
case 32: case 108:
return UNLIT | NO_VERTEX_COLORS | MATERIAL_COLOR;
// Basic NL, Basic NL UVAnim, OneSidedAlpha NL (and skinned), Opaque NL NoFog
case 34: case 36: case 56: case 61: case 83:
return UNLIT | NO_VERTEX_COLORS;
// Basic, the lit one without vertex colors
case 94:
return NO_VERTEX_COLORS;
// VertColor_NoLight_NoTex_AnimAlpha, VC_NL_NoTex_2D, Basic NL VC NT, OneSidedAlpha NL VC NT (and skinned),
// Basic NL NT, Opaque NL VC NT NoFog
case 11: case 16: case 33: case 58: case 63: case 80: case 82:
return UNLIT | NO_TEXTURE;
// Basic VC NT, Opaque VC NT NoFog
case 37: case 85:
return NO_TEXTURE;
// VertColor_NoLighting_Alpha, VertColorTex_NoLight_AlphaBlend and _AlphaTest, VC_NoLighting_2D, Over
// Everything (Unlit), Basic NL VC, LEGO-No Light, OneSidedAlpha NL VC (and skinned), OneSidedAlpha NL
// AnimAlpha, the NoLight scrolling UVs, Opaque NL VC NoFog, Additive NoLight VertColor, Two Textures Added NL
// VC AnimUV, Distortion (Ocean) Unlit, Two Layers Blended NL VC AnimUV
case 8: case 10: case 54: case 15: case 23: case 35: case 52: case 57: case 62: case 68: case 70: case 73: case 81:
case 84: case 87: case 93: case 101: case 105:
return UNLIT;
default:
return 0;
}
}
std::optional<Model> Parse(std::string_view data, uint32_t lod, std::string& error) {
return Parser(data, lod).Run(error);
}

16
dCommon/NifFile.h
View File

@@ -72,6 +72,22 @@ namespace NifFile {
// What a texture's alpha does under a shader (mapShaders.gameValue); -1 is fixed function (opacity)
eTextureAlpha TextureAlphaFor(int32_t shader);
/**
* What a shader (mapShaders.gameValue) leaves out of the usual lit look, which is (sun * max(0, N.L) + ambient)
* * vertex color * texture, the scene's lights from its .lvl, NiMaterialProperty's colors unused (the client's
* BasicShaders.fx, LEGOPPLighting.fx and Ocean.fx). Fixed function (-1) has none of these: Gamebryo lights it with
* the material and NiVertexColorProperty.
*/
enum eShaderLook : uint8_t {
UNLIT = 1, // no lighting: the colors as they are (the "NoLighting" techniques)
NO_TEXTURE = 2, // the texture isn't sampled ("NoTexture")
NO_VERTEX_COLORS = 4, // vertex colors aren't read
MATERIAL_COLOR = 8 // NiMaterialProperty's diffuse color is (the "Material" techniques)
};
// eShaderLook bits of a shader (mapShaders.gameValue); 0 for the usual lit look and for fixed function
uint8_t ShaderLookFor(int32_t shader);
// Where a node is in the model's space: p' = rotation * p + translation (row-major, scale folded in)
struct NodeTransform {
std::array<float, 9> rotation{ 1, 0, 0, 0, 1, 0, 0, 0, 1 };

34
dDashboardServer/routes/Scenery.cpp
View File

@@ -222,8 +222,14 @@ namespace {
};
for (const auto& [id, value] : g_ShaderValues) mode(value);
mode(NifFile::LEGO_SHADER);
// What each shader leaves out of the lit look (NifFile::eShaderLook bits), for the ones that do
nlohmann::json looks = nlohmann::json::object();
for (const auto& [id, value] : g_ShaderValues) {
if (const auto look = NifFile::ShaderLookFor(value)) looks[std::to_string(value)] = look;
}
manifest["shaderTags"] = std::move(tags);
manifest["textureAlpha"] = std::move(modes);
manifest["shaderLooks"] = std::move(looks);
manifest["multishader"] = NifFile::MULTISHADER;
manifest["defaultShader"] = NifFile::LEGO_SHADER;
}
@@ -234,6 +240,23 @@ namespace {
double Round(float value, double scale) { return std::round(static_cast<double>(value) * scale) / scale; }
/**
* Bump when NifFile's output changes: converted models kept on disk are made again, and the manifests' "format"
* goes into the viewers' model and texture URLs so browsers don't keep drawing the old ones (they're cached for
* a week). 2: meshes carry their multishader tag; conversions without it drew glom parts with the LEGO shader.
*/
constexpr uint32_t FORMAT_VERSION = 2;
// A zone's lighting (WorldScene::Lighting) for the viewers' shaders
nlohmann::json LightingJson(const WorldScene::Lighting& lighting) {
const auto triple = [](const std::array<float, 3>& value) { return nlohmann::json{ Round(value[0], 1000.0), Round(value[1], 1000.0), Round(value[2], 1000.0) }; };
return {
{"ambient", triple(lighting.ambient)}, {"light", triple(lighting.light)}, {"lightVec", triple(lighting.lightVec)},
{"upperHemi", triple(lighting.upperHemi)}, {"fogColor", triple(lighting.fogColor)},
{"fogNear", Round(lighting.fogNear, 10.0)}, {"fogFar", Round(lighting.fogFar, 10.0)}
};
}
/**
* A zone's models and scenery manifest, built once (g_Zones). Once shared, assets, index, flairModels and the
* warmed flags are guarded by g_ZoneMutex: the flairs' models join assets when their manifest is built.
@@ -242,6 +265,7 @@ namespace {
std::vector<std::string> assets; // res paths of models, indexed by the manifests
std::map<std::string, size_t> index; // res path -> its index in assets
std::string json;
nlohmann::json lighting; // LightingJson of the zone's lighting, null when its scenes have none
std::unordered_set<std::string> flairModels; // res paths of the flairs' models, converted ahead of others
bool warmedScenery{}; // WarmUp queued the scenery's models
bool warmedFlairs{}; // and the flairs'
@@ -266,9 +290,12 @@ namespace {
nlohmann::json hidden = nlohmann::json::array();
std::vector<int32_t> assetShaders;
int64_t sky = -1;
std::vector<std::pair<WorldScene::Lighting, size_t>> sceneLighting; // each scene's, with how many objects it has
for (const auto& scene : ZonePaths::ReadSceneFiles(*luz)) {
const auto lvl = ClientAssets::ReadResFile("maps/" + folder + scene);
if (!lvl) continue;
const auto objectsBefore = assetOf.size();
const auto lighting = WorldScene::ReadLighting(*lvl);
if (sky < 0) {
const auto skydome = JoinPath("", WorldScene::ReadSkydome(*lvl));
if (skydome.ends_with(".nif") && Files().paths.contains(skydome)) sky = static_cast<int64_t>(scenery.IndexOf(skydome));
@@ -286,9 +313,11 @@ namespace {
for (const auto value : { object.qx, object.qy, object.qz, object.qw }) rotations.push_back(Round(value, 10000.0));
scales.push_back(Round(object.scale, 1000.0));
}
if (lighting) sceneLighting.emplace_back(*lighting, assetOf.size() - objectsBefore);
}
if (const auto lighting = WorldScene::ZoneLighting(sceneLighting)) scenery.lighting = LightingJson(*lighting);
nlohmann::json manifest{
{"zone", zoneId}, {"sky", sky}, {"assets", scenery.assets},
{"zone", zoneId}, {"sky", sky}, {"assets", scenery.assets}, {"lighting", scenery.lighting}, {"format", FORMAT_VERSION},
{"objects", { {"asset", assetOf}, {"pos", positions}, {"rot", rotations}, {"scale", scales}, {"hidden", hidden} }}
};
assetShaders.resize(scenery.assets.size(), -1);
@@ -369,7 +398,7 @@ namespace {
assets = scenery.assets;
}
return nlohmann::json{
{"zone", zoneId}, {"sky", -1}, {"assets", assets}, {"distance", FLAIR_DISTANCE}, {"colorScale", 1.0 / 63.0},
{"zone", zoneId}, {"sky", -1}, {"assets", assets}, {"distance", FLAIR_DISTANCE}, {"colorScale", 1.0 / 63.0}, {"lighting", scenery.lighting}, {"format", FORMAT_VERSION},
{"objects", { {"asset", assetOf}, {"pos", positions}, {"rot", rotations}, {"scale", scales}, {"color", colors} }}
}.dump();
}
@@ -385,7 +414,6 @@ namespace {
return g_Flairs.Get(zoneId, [zoneId, &scenery] { return BuildFlairs(zoneId, scenery); });
}
constexpr uint32_t FORMAT_VERSION = 1; // bump when NifFile's output changes, so cached files are rebuilt
constexpr uintmax_t DISK_CACHE_BYTES = 512ull * 1024 * 1024;
const std::filesystem::path CACHE_DIR = std::filesystem::path("dDashboardServer") / "scenery_cache";

108
dDashboardServer/routes/WorldScene.h
View File

@@ -2,9 +2,11 @@
#include <algorithm>
#include <array>
#include <cmath>
#include <cstring>
#include <cstdint>
#include <exception>
#include <optional>
#include <sstream>
#include <string>
#include <vector>
@@ -121,6 +123,112 @@ namespace WorldScene {
return {};
}
/**
* A scene's lighting from its environment chunk, as the client's shaders get it (EnvironmentManager::SetLightEnv,
* 0x01088aa0 in client 1.10.64): g_ambientLight, g_lightColor (the sun's color), g_upperHemiLight and g_lightVec,
* the unit vector toward the sun (the file stores the direction the light shines in). Colors are 0..1.
*/
struct Lighting {
std::array<float, 3> ambient{};
std::array<float, 3> specular{};
std::array<float, 3> upperHemi{};
std::array<float, 3> light{}; // the sun's color
std::array<float, 3> lightVec{}; // toward the sun, unit length
std::array<float, 3> fogColor{};
float fogNear{}; // fog at the highest draw distance setting
float fogFar{};
bool operator==(const Lighting&) const = default;
};
/**
* The lighting of a scene file, read as the client's level_read_lighting_info (0x0102f8f0) does, gated by the
* file's version; nullopt when the file has none or ends early. Scenes older than version 36 have no sun color
* (the client keeps black there); ones before 31 no fog.
*/
inline std::optional<Lighting> ReadLighting(const std::string& lvl) {
std::istringstream stream(lvl);
LevelFile level;
try {
level.Read(stream);
} catch (const std::exception&) {
// The chunk headers read before any damage are enough
}
const auto info = level.chunkHeaders.find(LevelFile::FileInfo);
const auto environment = level.chunkHeaders.find(LevelFile::SceneEnviroment);
if (info == level.chunkHeaders.end() || environment == level.chunkHeaders.end()) return std::nullopt;
const auto version = info->second.fileInfo.version;
size_t at = environment->second.startPosition; // the chunk starts with the offset of its lighting
bool ok = true;
const auto u32 = [&]() {
uint32_t value{};
if (at > lvl.size() || lvl.size() - at < 4) {
ok = false;
return value;
}
std::memcpy(&value, lvl.data() + at, 4);
at += 4;
return value;
};
const auto f32 = [&]() {
const auto bits = u32();
float value{};
std::memcpy(&value, &bits, 4);
return value;
};
const auto read3 = [&](std::array<float, 3>& out) { for (auto& value : out) value = f32(); };
at = u32();
if (!ok || at == 0) return std::nullopt;
Lighting lighting;
if (version > 44) f32(); // how long the client blends to it
read3(lighting.ambient);
read3(lighting.specular);
read3(lighting.upperHemi);
std::array<float, 3> direction{};
read3(direction);
if (version > 30) {
if (version < 39) {
lighting.fogNear = f32();
lighting.fogFar = f32();
} else {
// Two draw distance settings (lowest, then highest): fog near and far, post fog solid and fade, static
// and dynamic object distance
for (int i = 0; i < 6; i++) f32();
lighting.fogNear = f32();
lighting.fogFar = f32();
for (int i = 0; i < 4; i++) f32();
if (version > 39) {
const auto cullGroups = u32(); // group id, min, max each
if (!ok || cullGroups > (lvl.size() - at) / 12) return std::nullopt;
at += static_cast<size_t>(cullGroups) * 12;
}
}
read3(lighting.fogColor);
}
if (version > 35) read3(lighting.light);
if (!ok) return std::nullopt;
const auto length = std::sqrt(direction[0] * direction[0] + direction[1] * direction[1] + direction[2] * direction[2]);
if (length > 0.0f) for (int i = 0; i < 3; i++) lighting.lightVec[i] = -direction[i] / length;
return lighting;
}
/**
* The lighting to draw a whole zone with: the client blends to each scene's lighting as the player walks in, so a
* view of the zone takes the one most of its objects are lit by (scenes as {lighting, how many objects}; ties go
* to the first). nullopt when no scene has any.
*/
inline std::optional<Lighting> ZoneLighting(const std::vector<std::pair<Lighting, size_t>>& scenes) {
std::vector<std::pair<Lighting, size_t>> totals;
for (const auto& [lighting, objects] : scenes) {
const auto it = std::find_if(totals.begin(), totals.end(), [&lighting](const auto& total) { return total.first == lighting; });
if (it == totals.end()) totals.emplace_back(lighting, objects);
else it->second += objects;
}
if (totals.empty()) return std::nullopt;
return std::max_element(totals.begin(), totals.end(), [](const auto& a, const auto& b) { return a.second < b.second; })->first;
}
/**
* The kinds objects are drawn as, most telling first: an enemy that also offers missions is an enemy. Anything with
* none of these components is "other" (scenery, volumes, markers). Names come from the enum (GameLabels).

48
dDashboardServer/static/js/scenery-core.js
View File

@@ -43,14 +43,46 @@ export function parseModel(buffer) {
* their shader in their node names (mesh.shaderTag); a tag the client can't use falls back to the LEGO shader.
*/
export function textureAlphaMode(manifest, asset, mesh) {
if (!manifest || !manifest.shaders || !manifest.textureAlpha) return 'opacity';
let shader = manifest.shaders[asset];
if (shader === undefined || shader === null) return 'opacity';
if (shader === manifest.multishader) {
const tagged = mesh && mesh.shaderTag >= 0 && manifest.shaderTags ? manifest.shaderTags[mesh.shaderTag] : undefined;
shader = tagged !== undefined && tagged >= 3 && tagged <= 108 ? tagged : manifest.defaultShader;
}
return manifest.textureAlpha[shader] || 'opacity';
if (!manifest || !manifest.textureAlpha) return 'opacity';
const shader = shaderOf(manifest, asset, mesh);
return shader === null ? 'opacity' : manifest.textureAlpha[shader] || 'opacity';
}
/**
* The shader (mapShaders.gameValue) the game draws a mesh of a model with, or null when the manifest doesn't say
* (-1 is fixed function). A multishader model's parts name theirs in their node names (mesh.shaderTag, a mapShaders
* id); a tag the client can't use falls back to the LEGO shader.
*/
export function shaderOf(manifest, asset, mesh) {
if (!manifest || !manifest.shaders) return null;
const shader = manifest.shaders[asset];
if (shader === undefined || shader === null) return null;
if (shader !== manifest.multishader) return shader;
const tagged = mesh && mesh.shaderTag >= 0 && manifest.shaderTags ? manifest.shaderTags[mesh.shaderTag] : undefined;
return tagged !== undefined && tagged >= 3 && tagged <= 108 ? tagged : manifest.defaultShader;
}
// NifFile::eShaderLook bits
export const SHADER_LOOK = { UNLIT: 1, NO_TEXTURE: 2, NO_VERTEX_COLORS: 4, MATERIAL_COLOR: 8 };
/**
* How a mesh is drawn under the game's shaders, when the manifest has the zone's lighting: {lit, texture,
* vertexColors, material} — whether the scene's sun and ambient light it, its texture and vertex colors are used, and
* whether its NiMaterialProperty colors are (only fixed function and the "Material" shaders use them). Null without
* lighting in the manifest (older servers), for the viewer's own lights.
*/
export function gameLook(manifest, asset, mesh) {
if (!manifest || !manifest.lighting) return null;
const shader = shaderOf(manifest, asset, mesh);
const fixedFunction = shader === null || shader < 0;
const bits = fixedFunction || !manifest.shaderLooks ? 0 : manifest.shaderLooks[shader] || 0;
return {
lit: !(bits & SHADER_LOOK.UNLIT),
texture: !(bits & SHADER_LOOK.NO_TEXTURE),
// Fixed function reads them as NiVertexColorProperty says; the shaders always do, unless they have none
vertexColors: !!(mesh.colors && !(bits & SHADER_LOOK.NO_VERTEX_COLORS) && (!fixedFunction || mesh.vertexColors !== 0)),
material: fixedFunction || !!(bits & SHADER_LOOK.MATERIAL_COLOR)
};
}
/**

104
dDashboardServer/static/js/scenery.js
View File

@@ -9,7 +9,7 @@
* follows the camera, far away, behind everything.
*/
import * as THREE from 'three';
import { parseModel, mergeMeshes, parseDds, decodeDxt, completeChain, linearColors, groupObjects, cellsOf, textureAlphaMode } from '/js/scenery-core.js';
import { parseModel, mergeMeshes, parseDds, decodeDxt, completeChain, linearColors, groupObjects, cellsOf, textureAlphaMode, gameLook } from '/js/scenery-core.js';
// Per detail level (the property view's 0 high, 1 medium, 2 low): the model LOD, how far objects are drawn, the
// largest texture side and a memory budget for geometry and textures
@@ -64,6 +64,12 @@ export function createScenery({ scene, camera, renderer, urls, focus, onProgress
onProgress(done, wanted);
}
// A model or texture URL with the manifest's conversion format, so a browser's week-long cache of models
// converted the old way isn't used once the server converts them anew
function versioned(url) {
return manifest && manifest.format ? url + (url.includes('?') ? '&' : '?') + 'v=' + manifest.format : url;
}
async function fetchBuffer(url) {
const response = await fetch(url, { credentials: 'same-origin' });
if (!response.ok) throw new Error(response.status + ' ' + url);
@@ -105,7 +111,7 @@ export function createScenery({ scene, camera, renderer, urls, focus, onProgress
const key = lod + '/' + (path.startsWith('#') ? asset + path : path);
if (!textures.has(key)) {
const size = detail.texture;
textures.set(key, fetchBuffer(urls.texture(manifest.zone, asset, slot, lod)).then((buffer) => {
textures.set(key, fetchBuffer(versioned(urls.texture(manifest.zone, asset, slot, lod))).then((buffer) => {
const texture = makeTexture(buffer, size);
if (texture) used += texture.userData.bytes;
return texture;
@@ -114,12 +120,15 @@ export function createScenery({ scene, camera, renderer, urls, focus, onProgress
return textures.get(key);
}
function geometryOf(mesh) {
// Whether a mesh's vertex colors are drawn: as its game look says, else as NiVertexColorProperty does
const usesVertexColors = (mesh, look) => (look ? look.vertexColors : !!(mesh.colors && mesh.vertexColors !== 0));
function geometryOf(mesh, look) {
const geometry = new THREE.BufferGeometry();
geometry.setAttribute('position', new THREE.BufferAttribute(mesh.positions, 3));
if (mesh.normals) geometry.setAttribute('normal', new THREE.BufferAttribute(mesh.normals, 3, true));
if (mesh.uvs) geometry.setAttribute('uv', new THREE.BufferAttribute(mesh.uvs, 2));
if (mesh.colors && mesh.vertexColors !== 0) geometry.setAttribute('color', new THREE.BufferAttribute(linearColors(mesh.colors), 4, true));
if (usesVertexColors(mesh, look)) geometry.setAttribute('color', new THREE.BufferAttribute(linearColors(mesh.colors), 4, true));
geometry.setIndex(new THREE.BufferAttribute(mesh.indices, 1));
if (!mesh.normals) geometry.computeVertexNormals();
geometry.computeBoundingSphere();
@@ -149,51 +158,105 @@ export function createScenery({ scene, camera, renderer, urls, focus, onProgress
#endif
`;
// The texture alpha mode's change to a fragment shader
function textureAlphaPatch(shader, mode) {
if (mode === 'decal') {
shader.fragmentShader = shader.fragmentShader.replace('#include <map_fragment>', '').replace('#include <color_fragment>', DECAL_FRAGMENT);
} else if (mode === 'ignored') {
shader.fragmentShader = shader.fragmentShader.replace('#include <map_fragment>', OPAQUE_MAP_FRAGMENT);
}
}
function useTextureAlpha(material, mode) {
if (mode === 'opacity' || !material.map) return material;
material.onBeforeCompile = (shader) => {
if (mode === 'decal') {
shader.fragmentShader = shader.fragmentShader.replace('#include <map_fragment>', '').replace('#include <color_fragment>', DECAL_FRAGMENT);
} else {
shader.fragmentShader = shader.fragmentShader.replace('#include <map_fragment>', OPAQUE_MAP_FRAGMENT);
}
};
material.onBeforeCompile = (shader) => textureAlphaPatch(shader, mode);
material.customProgramCacheKey = () => 'textureAlpha:' + mode;
return material;
}
function materialOf(mesh, map, forSky, alphaMode = 'opacity') {
// The zone's lights as the game's shaders get them (manifest.lighting), shared by every lit material
const gameLights = { gameLightColor: { value: new THREE.Vector3(1, 1, 1) }, gameAmbient: { value: new THREE.Vector3() }, gameLightVec: { value: new THREE.Vector3(0, 1, 0) } };
function setGameLights(lighting) {
if (!lighting) return;
gameLights.gameLightColor.value.fromArray(lighting.light);
gameLights.gameAmbient.value.fromArray(lighting.ambient);
gameLights.gameLightVec.value.fromArray(lighting.lightVec);
}
/**
* Lighting per vertex as BasicShaders.fx and LEGOPPLighting.fx do it: sun * max(0, N.L) + ambient in the game's
* (sRGB) color space, which the vertex shader's color output clamps to 1, then made linear for three.js.
*/
const GAME_LIGHT_VERTEX = `#include <begin_vertex>
vec3 gameNormal = normal;
#ifdef USE_INSTANCING
gameNormal = mat3( instanceMatrix ) * gameNormal;
#endif
gameNormal = normalize( mat3( modelMatrix ) * gameNormal );
vGameLight = pow( clamp( gameLightColor * max( 0.0, dot( gameNormal, gameLightVec ) ) + gameAmbient, 0.0, 1.0 ), vec3( 2.2 ) );`;
/**
* A material that draws a mesh the way its game shader does (gameLook): unlit by the view's own lights and tone
* mapping, the zone's sun and ambient light per vertex when the shader is lit, the material's color only when the
* shader reads it.
*/
function gameMaterial(options, mesh, alphaMode, look) {
const material = new THREE.MeshBasicMaterial({
...options,
color: look.material ? options.color : new THREE.Color(1, 1, 1)
});
material.toneMapped = false;
material.onBeforeCompile = (shader) => {
if (options.map) textureAlphaPatch(shader, alphaMode);
if (!look.lit) return;
Object.assign(shader.uniforms, gameLights);
shader.vertexShader = 'uniform vec3 gameLightColor;\nuniform vec3 gameAmbient;\nuniform vec3 gameLightVec;\nvarying vec3 vGameLight;\n' +
shader.vertexShader.replace('#include <begin_vertex>', GAME_LIGHT_VERTEX);
shader.fragmentShader = 'varying vec3 vGameLight;\n' +
shader.fragmentShader.replace('#include <aomap_fragment>', '#include <aomap_fragment>\n\treflectedLight.indirectDiffuse *= vGameLight;');
};
material.customProgramCacheKey = () => 'game:' + (options.map ? alphaMode : '') + ':' + look.lit;
return material;
}
function materialOf(mesh, map, forSky, alphaMode = 'opacity', look = null) {
// Nearly everything in the game's files has alpha blending switched on; it only shows where something is see-
// through: the material, a vertex or the texture (only when the object's shader uses the texture's alpha as
// opacity). Blended meshes still write depth, as Gamebryo's default does.
const vertexColors = usesVertexColors(mesh, look);
let vertexAlpha = false;
if (mesh.colors && mesh.vertexColors !== 0) for (let i = 3; i < mesh.colors.length && !vertexAlpha; i += 4) vertexAlpha = mesh.colors[i] < 250;
if (vertexColors) for (let i = 3; i < mesh.colors.length && !vertexAlpha; i += 4) vertexAlpha = mesh.colors[i] < 250;
const textureAlpha = alphaMode === 'opacity' && !!(map && map.userData.alpha);
const seeThrough = mesh.blend && (mesh.alpha < 0.99 || vertexAlpha || textureAlpha);
// The game's shaders take alpha from the vertex colors and texture only; NiMaterialProperty's is for fixed function
const materialAlpha = look && !look.material ? 1 : mesh.alpha;
const seeThrough = mesh.blend && (materialAlpha < 0.99 || vertexAlpha || textureAlpha);
const options = {
color: new THREE.Color().setRGB(mesh.diffuse[0], mesh.diffuse[1], mesh.diffuse[2], THREE.SRGBColorSpace),
vertexColors: !!(mesh.colors && mesh.vertexColors !== 0),
vertexColors,
transparent: seeThrough,
opacity: mesh.alpha,
opacity: materialAlpha,
alphaTest: mesh.test >= 0 ? Math.max(mesh.test / 255, 0.01) : 0,
side: mesh.doubleSided ? THREE.DoubleSide : THREE.FrontSide,
map: map || null
};
if (forSky) return useTextureAlpha(new THREE.MeshBasicMaterial({ ...options, depthWrite: false, fog: false }), alphaMode);
if (look) return gameMaterial(options, mesh, alphaMode, look);
const material = new THREE.MeshStandardMaterial({ ...options, roughness: 0.85, metalness: 0 });
material.emissive.setRGB(mesh.emissive[0], mesh.emissive[1], mesh.emissive[2], THREE.SRGBColorSpace);
return useTextureAlpha(material, alphaMode);
}
async function buildParts(asset, lod, forSky) {
const buffer = await fetchBuffer(urls.mesh(manifest.zone, asset, lod));
const buffer = await fetchBuffer(versioned(urls.mesh(manifest.zone, asset, lod)));
const model = parseModel(buffer);
const parts = [];
// The sky's layers keep their order; everything else has its look-alike pieces joined
for (const mesh of forSky ? model.meshes : mergeMeshes(model.meshes)) {
if (!mesh.vertices || !mesh.indices.length) continue;
// The sky keeps its own unlit look
const look = forSky ? null : gameLook(manifest, asset, mesh);
let map = null;
if (mesh.texture >= 0 && mesh.uv) {
if (mesh.texture >= 0 && mesh.uv && (!look || look.texture)) {
const texture = await loadTexture(asset, mesh.texture, model.header.textures[mesh.texture], lod);
if (texture) {
map = texture.clone(); // shares the image; wrapping differs per mesh
@@ -202,8 +265,8 @@ export function createScenery({ scene, camera, renderer, urls, focus, onProgress
map.needsUpdate = true;
}
}
const geometry = geometryOf(mesh);
parts.push({ geometry, material: materialOf(mesh, map, forSky, textureAlphaMode(manifest, asset, mesh)) });
const geometry = geometryOf(mesh, look);
parts.push({ geometry, material: materialOf(mesh, map, forSky, textureAlphaMode(manifest, asset, mesh), look) });
}
const min = model.header.min, max = model.header.max;
const radius = Math.hypot(max[0] - min[0], max[1] - min[1], max[2] - min[2]) / 2;
@@ -361,6 +424,7 @@ export function createScenery({ scene, camera, renderer, urls, focus, onProgress
if (!response.ok) { manifest = null; return false; }
manifest = await response.json();
manifest.url = url;
setGameLights(manifest.lighting);
byAsset = groupObjects(manifest.objects);
if (manifest.sky >= 0) byAsset.delete(manifest.sky);
}

7
tests/dWebTests/CMakeLists.txt
View File

@@ -82,3 +82,10 @@ target_link_libraries(dWebTests ${COMMON_LIBRARIES} dWeb dUgc sqlite3 OpenSSL::C
gtest_discover_tests(dWebTests)
target_compile_definitions(dWebTests PRIVATE DLU_SOURCE_DIR="${PROJECT_SOURCE_DIR}")
# The 3D views' shader lookups (static/js/scenery-core.js), when node is there
find_program(NODE_EXECUTABLE node)
if(NODE_EXECUTABLE)
add_test(NAME SceneryCoreJs COMMAND ${NODE_EXECUTABLE} "${CMAKE_CURRENT_SOURCE_DIR}/scenery-core.test.mjs"
"${PROJECT_SOURCE_DIR}/dDashboardServer/static/js/scenery-core.js")
endif()

87
tests/dWebTests/NifFileTests.cpp
View File

@@ -355,6 +355,93 @@ TEST(WorldSceneTests, ReadsTheSkydomeOfASceneFile) {
EXPECT_EQ(WorldScene::ReadSkydome(""), "");
}
namespace {
// A scene file (`version`) with one environment chunk whose lighting is `lighting`
std::string SceneWithLighting(uint32_t version, const Bytes& lighting) {
Bytes lvl;
// File info chunk (data at 0x20: version), then the environment chunk (header at 0x34, data at 0x54) whose
// lighting starts at 0x60
lvl.Raw("CHNK").Put<uint32_t>(1000).Put<uint16_t>(1).Put<uint16_t>(1).Put<uint32_t>(0x34).Put<uint32_t>(0x20);
lvl.Raw(std::string(0x20 - lvl.data.size(), '\xCD'));
lvl.Put<uint32_t>(version).Put<uint32_t>(0).Put<uint32_t>(0x34).Put<uint32_t>(0).Put<uint32_t>(0);
lvl.Raw("CHNK").Put<uint32_t>(2000).Put<uint16_t>(1).Put<uint16_t>(2).Put<uint32_t>(0x60 + static_cast<uint32_t>(lighting.data.size()) - 0x34).Put<uint32_t>(0x54);
lvl.Raw(std::string(0x54 - lvl.data.size(), '\xCD'));
lvl.Put<uint32_t>(0x60).Put<uint32_t>(0).Put<uint32_t>(0);
lvl.Raw(lighting.data);
return lvl.data;
}
Bytes& Floats(Bytes& bytes, std::initializer_list<float> values) {
for (const auto value : values) bytes.Put<float>(value);
return bytes;
}
}
// As the client's level_read_lighting_info: version 48 has a blend time, two draw distance settings and cull groups
TEST(WorldSceneTests, ReadsTheLightingOfASceneFile) {
Bytes lighting;
Floats(lighting, { 10.0f }); // blend time
Floats(lighting, { 0.42f, 0.62f, 0.75f }); // ambient
Floats(lighting, { 1, 1, 1 }); // specular
Floats(lighting, { 1, 0.7f, 0.5f }); // upper hemisphere
Floats(lighting, { 0, -3, -4 }); // the way the sun shines
Floats(lighting, { 100, 300, 50, 50, 8000, 8000 }); // lowest draw distances
Floats(lighting, { 250, 350, 100, 100, 8000, 8000 }); // highest
lighting.Put<uint32_t>(2).Put<uint32_t>(7);
Floats(lighting, { 1, 2 });
lighting.Put<uint32_t>(8);
Floats(lighting, { 3, 4 }); // cull groups
Floats(lighting, { 0.5f, 0.8f, 0.9f }); // fog color
Floats(lighting, { 1, 0.9f, 0.8f }); // sun color
const auto read = WorldScene::ReadLighting(SceneWithLighting(48, lighting));
ASSERT_TRUE(read);
EXPECT_FLOAT_EQ(read->ambient[1], 0.62f);
EXPECT_FLOAT_EQ(read->upperHemi[2], 0.5f);
EXPECT_FLOAT_EQ(read->lightVec[0], 0.0f); // toward the sun: the stored direction turned around, unit length
EXPECT_FLOAT_EQ(read->lightVec[1], 0.6f);
EXPECT_FLOAT_EQ(read->lightVec[2], 0.8f);
EXPECT_FLOAT_EQ(read->fogNear, 250.0f);
EXPECT_FLOAT_EQ(read->fogFar, 350.0f);
EXPECT_FLOAT_EQ(read->fogColor[2], 0.9f);
EXPECT_FLOAT_EQ(read->light[1], 0.9f);
// Version 35: no blend time, one fog range, no sun color
Bytes old;
Floats(old, { 0.5f, 0.5f, 0.5f, 1, 1, 1, 1, 1, 1, 0, -1, 0, 20, 90, 0.1f, 0.2f, 0.3f });
const auto older = WorldScene::ReadLighting(SceneWithLighting(35, old));
ASSERT_TRUE(older);
EXPECT_FLOAT_EQ(older->ambient[0], 0.5f);
EXPECT_FLOAT_EQ(older->lightVec[1], 1.0f);
EXPECT_FLOAT_EQ(older->fogFar, 90.0f);
EXPECT_FLOAT_EQ(older->fogColor[1], 0.2f);
EXPECT_FLOAT_EQ(older->light[0], 0.0f);
// Cut short, or no environment chunk
const auto whole = SceneWithLighting(48, lighting);
EXPECT_FALSE(WorldScene::ReadLighting(whole.substr(0, whole.size() - 8)));
EXPECT_FALSE(WorldScene::ReadLighting(""));
}
TEST(WorldSceneTests, LightsAZoneAsMostOfItsObjectsAre) {
WorldScene::Lighting day, dusk;
day.ambient = { 1, 1, 1 };
dusk.ambient = { 0.2f, 0.2f, 0.4f };
EXPECT_FALSE(WorldScene::ZoneLighting({}));
// Two scenes lit like dusk hold more objects than the day one
EXPECT_EQ(WorldScene::ZoneLighting({ { day, 50 }, { dusk, 30 }, { dusk, 25 } }), dusk);
EXPECT_EQ(WorldScene::ZoneLighting({ { day, 10 }, { dusk, 10 } }), day); // a tie goes to the first
}
TEST(NifFileTests, KnowsWhatEachShaderLeavesOut) {
EXPECT_EQ(NifFile::ShaderLookFor(38), 0); // Basic VC: lit, textured, vertex colors
EXPECT_EQ(NifFile::ShaderLookFor(NifFile::LEGO_SHADER), 0);
EXPECT_EQ(NifFile::ShaderLookFor(-1), 0); // fixed function is lit by Gamebryo
EXPECT_EQ(NifFile::ShaderLookFor(33), NifFile::UNLIT | NifFile::NO_TEXTURE); // Basic NL VC NT
EXPECT_EQ(NifFile::ShaderLookFor(37), NifFile::NO_TEXTURE); // Basic VC NT
EXPECT_EQ(NifFile::ShaderLookFor(70), NifFile::UNLIT); // ScrollingUV_NoLight_AnimAlpha
EXPECT_EQ(NifFile::ShaderLookFor(32), NifFile::UNLIT | NifFile::NO_VERTEX_COLORS | NifFile::MATERIAL_COLOR); // Basic NL Material
}
// The game client's own meshes, when a client is configured (DLU_CLIENT_RES, else client_location in the build's
// sharedconfig.ini): the first 300 .nif files under res/mesh/env read, and most have something to draw
TEST(NifFileTests, ReadsTheClientsMeshes) {

50
tests/dWebTests/scenery-core.test.mjs Normal file
View File

@@ -0,0 +1,50 @@
// The 3D views' shader lookups (static/js/scenery-core.js): which shader draws a mesh and what it uses.
// Run by ctest: node scenery-core.test.mjs <scenery-core.js>
import { pathToFileURL } from 'node:url';
const [modulePath] = process.argv.slice(2);
const S = await import(pathToFileURL(modulePath).href);
let failures = 0;
const same = (actual, expected, what) => {
if (JSON.stringify(actual) !== JSON.stringify(expected)) {
failures++;
console.error(`${what}: ${JSON.stringify(actual)} is not ${JSON.stringify(expected)}`);
}
};
const manifest = {
shaders: [38, 9999, -1, 33],
shaderTags: { 1: 5, 30: 38, 25: 33, 2: 2 },
textureAlpha: { 5: 'decal' },
shaderLooks: { 33: S.SHADER_LOOK.UNLIT | S.SHADER_LOOK.NO_TEXTURE },
multishader: 9999,
defaultShader: 5,
lighting: { ambient: [0.4, 0.6, 0.7], light: [1, 1, 1], lightVec: [0, 1, 0] }
};
const colored = { colors: new Uint8Array(4), vertexColors: 2 };
// A multishader part's tag names its shader; an unusable or missing tag is the LEGO shader
same(S.shaderOf(manifest, 1, { shaderTag: 30 }), 38, 'tagged part');
same(S.shaderOf(manifest, 1, { shaderTag: 2 }), 5, 'unusable tag');
same(S.shaderOf(manifest, 1, { shaderTag: -1 }), 5, 'untagged part');
same(S.shaderOf(manifest, 0, {}), 38, 'object shader');
same(S.shaderOf({}, 0, {}), null, 'no shaders in the manifest');
same(S.textureAlphaMode(manifest, 1, { shaderTag: 1 }), 'decal', 'LEGO part texture alpha');
same(S.textureAlphaMode(manifest, 1, { shaderTag: 30 }), 'opacity', 'Basic VC part texture alpha');
// Lit, textured, vertex colors, no material colors: Basic VC
same(S.gameLook(manifest, 1, { ...colored, shaderTag: 30 }), { lit: true, texture: true, vertexColors: true, material: false }, 'Basic VC');
// Vertex colors are read even when NiVertexColorProperty ignores them, but only if the mesh has some
same(S.gameLook(manifest, 0, { ...colored, vertexColors: 0 }).vertexColors, true, 'shader reads vertex colors');
same(S.gameLook(manifest, 0, {}).vertexColors, false, 'mesh without vertex colors');
same(S.gameLook(manifest, 3, colored), { lit: false, texture: false, vertexColors: true, material: false }, 'Basic NL VC NT');
// Fixed function: NiVertexColorProperty and the material decide
same(S.gameLook(manifest, 2, { ...colored, vertexColors: 0 }), { lit: true, texture: true, vertexColors: false, material: true }, 'fixed function');
// Without the zone's lighting the viewer lights scenery itself
same(S.gameLook({ ...manifest, lighting: null }, 0, colored), null, 'no lighting');
if (failures) {
console.error(`${failures} failed`);
process.exit(1);
}
console.log('scenery-core: all passed');