feat: Raw reading on zone load and scene tools (#1910)

* feat: raw terrain parsing for scene data

Replace old dNavigation/dTerrain raw parser with new Raw module in
dZoneManager. Parse heightmaps, color maps, and scene maps from .raw
files to determine which scene a position belongs to. Build scene
adjacency graph from terrain data and scene transitions.

Adds NiColor type, SceneColor lookup table, eSceneType enum, terrain
mesh generation with OBJ export, and debug slash commands for scene
visualization.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: deduplicate terrain grid lookup logic, remove unnecessary cast

Add IsValidForSceneLookup(), GetSceneIDAtGrid(), and GridToWorldPos()
to Raw::Chunk. Use them in GenerateTerrainMesh, GetSceneIDFromPosition,
SpawnScenePoints, and SpawnAllScenePoints instead of duplicated inline
math. Remove unnecessary static_cast in LoadSceneTransitionInfo.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: add version < 32 support to raw terrain parser

The parser was only correct for v32 files. For v<32:
- colorMapResolution is width-1, not width
- Color map reads width*width*4 bytes (BGRA per-pixel), not colorMapRes^2*4
- No DDS lightmap
- No texture settings byte or blend DDS
- Scene map: v31 reads (colorMapRes+1)^2 cells with border skip,
  v<31 skips 1 byte
- No mesh data

Verified against all 309 .raw files across multiple client versions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: log and reject legacy raw terrain format, clean up ReadRaw API

Log a clear message and throw if the zone uses a legacy raw format
(LUZ version < 30) since no known maps use it. Remove ReadLegacyRaw
and the mapVersion parameter from ReadRaw.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: revert v33 scene field renames to unknown1/unknown2

These fields are skipped by the client in every known build (ptr += 12
and ptr += 4). No client ever reads them into a struct. Keep the
original unknown1/unknown2 names from main.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: load the zone without scene lookups when its raw terrain is missing or unreadable

Log at debug when it parses; build the terrain mesh only for the OBJ export; read the zone name and description only for version 31 and later, as the client does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: look up the scene at a position as the client does, in constant time

Chunk bounds span (width - 1) * scale; the nearest scene map cell is used and the far edge is kept one cell inside. Chunks are indexed by grid cell when they form a grid. Skipping reads instead of seeking, so a cut-off file can't seek past the end. Terrain mesh indices follow the vertex order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat: scene point commands take a step and use brick LOTs from the BrickIDTable

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: rename NiColor::ToXYZ to Luminance

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: raw terrain reading, scene lookup, scene graph and scene commands

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: close the OBJ file before removing it so the test passes on Windows

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: keep the zone boundary lines and read the luz scene count as the client does

Each boundary line's destination is read as a map ID and an instance ID into a LWOZONEID. The scene count is a byte before version 37 and a u32 from it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: name the fourth flair colour byte padding

The client never sets or reads it, so it holds leftover memory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: keep the scene map of version 31 and older raws

These files have a (resolution + 1)^2 scene map, kept in sceneMapLegacy. Version 31 uses its first resolution rows and columns; older versions are all scene 0, as in the client.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor: simplify the raw heightmap size check

The dimensions are capped at kMaxResolution, so the product can't overflow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: remove the scene point spawn commands and send /getadjacentscenes as one message

The spawn commands were only for testing. /getscene and /getadjacentscenes stay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Aaron Kimbrell
2026-10-27 09:22:28 -05:00
committed by GitHub
parent fc6a7fbd52
commit 9bf2ff4d44
29 changed files with 2102 additions and 315 deletions

View File

@@ -8,6 +8,9 @@ list(APPEND DGAMETEST_SOURCES ${DCOMPONENTS_TESTS})
add_subdirectory(dGameMessagesTests)
list(APPEND DGAMETEST_SOURCES ${DGAMEMESSAGES_TESTS})
add_subdirectory(dZoneManagerTests)
list(APPEND DGAMETEST_SOURCES ${DZONEMANAGER_TESTS})
file(COPY ${GAMEMESSAGE_TESTBITSTREAMS} DESTINATION ${CMAKE_CURRENT_BINARY_DIR})
file(COPY ${COMPONENT_TEST_DATA} DESTINATION ${CMAKE_CURRENT_BINARY_DIR})

View File

@@ -0,0 +1,10 @@
SET(DZONEMANAGER_TESTS
"RawTests.cpp"
"ZoneSceneTests.cpp")
# Get the folder name and prepend it to the files above
get_filename_component(thisFolderName ${CMAKE_CURRENT_SOURCE_DIR} NAME)
list(TRANSFORM DZONEMANAGER_TESTS PREPEND "${thisFolderName}/")
# Export to parent scope
set(DZONEMANAGER_TESTS ${DZONEMANAGER_TESTS} PARENT_SCOPE)

View File

@@ -0,0 +1,381 @@
#include <gtest/gtest.h>
#include <filesystem>
#include <fstream>
#include <random>
#include "Game.h"
#include "Logger.h"
#include "NiColor.h"
#include "Raw.h"
#include "SceneColor.h"
#include "TerrainTestWriter.h"
using namespace TerrainTestWriter;
class RawTest : public ::testing::Test {
protected:
void SetUp() override {
Game::logger = new Logger("./testing.log", false, false);
}
void TearDown() override {
delete Game::logger;
Game::logger = nullptr;
}
static bool Read(const std::string& data, Raw::Raw& raw) {
std::istringstream stream(data);
return Raw::ReadRaw(stream, raw);
}
// A chunksX by chunksZ grid of 5x5 point chunks, scale 2 (8 units across), each with a 4x4 scene map whose cells
// are numbered [x * 4 + z] plus 16 * the chunk's index
static std::vector<TestChunk> MakeGrid(uint32_t chunksX, uint32_t chunksZ, float originX = -8.0f, float originZ = -8.0f) {
std::vector<TestChunk> chunks;
for (uint32_t x = 0; x < chunksX; ++x) {
for (uint32_t z = 0; z < chunksZ; ++z) {
TestChunk chunk;
chunk.id = x * chunksZ + z;
chunk.width = 5;
chunk.height = 5;
chunk.scale = 2.0f;
chunk.offsetX = originX + 8.0f * x;
chunk.offsetZ = originZ + 8.0f * z;
chunk.colorMapResolution = 4;
for (uint8_t cell = 0; cell < 16; ++cell) chunk.sceneMap.push_back(static_cast<uint8_t>(cell + 16 * chunk.id));
chunks.push_back(chunk);
}
}
return chunks;
}
};
TEST_F(RawTest, ReadsEveryPartOfVersion32) {
auto chunks = MakeGrid(2, 3);
chunks[1].vertSize = 0; // a chunk without a render mesh
chunks[2].flairCount = 0;
const auto data = WriteRaw(chunks, 32, 2, 3);
std::istringstream stream(data);
Raw::Raw raw;
ASSERT_TRUE(Raw::ReadRaw(stream, raw));
EXPECT_EQ(stream.tellg(), static_cast<std::streamoff>(data.size())); // every block was read or skipped
EXPECT_EQ(raw.version, 32);
EXPECT_EQ(raw.dev, 0);
EXPECT_EQ(raw.numChunks, 6);
EXPECT_EQ(raw.numChunksWidth, 3);
EXPECT_EQ(raw.numChunksHeight, 2);
ASSERT_EQ(raw.chunks.size(), 6);
const auto& chunk = raw.chunks[4];
EXPECT_EQ(chunk.id, 4);
EXPECT_EQ(chunk.width, 5);
EXPECT_EQ(chunk.height, 5);
EXPECT_FLOAT_EQ(chunk.offsetX, 0.0f);
EXPECT_FLOAT_EQ(chunk.offsetZ, 0.0f);
EXPECT_FLOAT_EQ(chunk.scaleFactor, 2.0f);
ASSERT_EQ(chunk.heightMap.size(), 25);
EXPECT_FLOAT_EQ(chunk.heightMap[7], 7.0f);
EXPECT_EQ(chunk.colorMapResolution, 4);
ASSERT_EQ(chunk.sceneMap.size(), 16);
EXPECT_EQ(chunk.sceneMap[5], 5 + 16 * 4);
EXPECT_TRUE(chunk.IsValidForSceneLookup());
EXPECT_FLOAT_EQ(chunk.sceneCellSizeX, 2.0f);
EXPECT_FLOAT_EQ(chunk.sceneCellsPerUnitZ, 0.5f);
EXPECT_EQ(chunk.textureIds, (std::vector<uint32_t>{ 100, 101, 102, 103 }));
EXPECT_EQ(chunk.colorMap.size(), 4 * 4 * 4);
EXPECT_EQ(chunk.colorMap[0], 0xAB);
EXPECT_EQ(chunk.lightMap.size(), 5);
EXPECT_EQ(chunk.textureMapResolution, 2);
EXPECT_EQ(chunk.textureMap.size(), 2 * 2 * 4);
EXPECT_EQ(chunk.textureSettings, 1);
EXPECT_EQ(chunk.blendMap.size(), 7);
ASSERT_EQ(chunk.flairs.size(), 2);
const auto& flair = chunk.flairs[1];
EXPECT_EQ(flair.id, 501);
EXPECT_FLOAT_EQ(flair.scaleFactor, 1.5f);
EXPECT_FLOAT_EQ(flair.position.x, 2.0f);
EXPECT_FLOAT_EQ(flair.position.z, 3.0f);
EXPECT_FLOAT_EQ(flair.rotation.y, 90.0f);
EXPECT_EQ(flair.colorR, 10);
EXPECT_EQ(flair.colorG, 20);
EXPECT_EQ(flair.colorB, 30);
EXPECT_EQ(flair.padding, 40);
EXPECT_TRUE(raw.chunks[2].flairs.empty());
EXPECT_EQ(chunk.vertSize, 4);
EXPECT_EQ(chunk.meshVertUsage.size(), 4);
EXPECT_EQ(chunk.meshVertSize.size(), 16);
ASSERT_EQ(chunk.meshTri.size(), 16);
EXPECT_EQ(chunk.meshTri[2].meshTriListSize, 6);
EXPECT_EQ(chunk.meshTri[2].meshTriList.size(), 6);
EXPECT_EQ(chunk.meshTri[15].meshTriListSize, 3);
EXPECT_EQ(raw.chunks[1].vertSize, 0);
EXPECT_TRUE(raw.chunks[1].meshTri.empty());
}
TEST_F(RawTest, BoundsAndChunkGridUseTheSpanBetweenFirstAndLastPoint) {
Raw::Raw raw;
ASSERT_TRUE(Read(WriteRaw(MakeGrid(2, 3), 32, 2, 3), raw));
// 5 points 2 apart span 8 units, so chunks don't overlap
EXPECT_FLOAT_EQ(raw.minBoundsX, -8.0f);
EXPECT_FLOAT_EQ(raw.maxBoundsX, 8.0f);
EXPECT_FLOAT_EQ(raw.minBoundsZ, -8.0f);
EXPECT_FLOAT_EQ(raw.maxBoundsZ, 16.0f);
EXPECT_FLOAT_EQ(raw.chunks[0].GetMaxX(), 0.0f);
EXPECT_EQ(raw.chunkGridCountX, 2);
EXPECT_EQ(raw.chunkGridCountZ, 3);
EXPECT_FLOAT_EQ(raw.chunkSpanX, 8.0f);
EXPECT_FLOAT_EQ(raw.chunkSpanZ, 8.0f);
ASSERT_EQ(raw.chunkGrid.size(), 6);
for (int32_t i = 0; i < 6; ++i) EXPECT_EQ(raw.chunkGrid[i], i);
}
TEST_F(RawTest, ChunksOutOfOrderAreFoundThroughTheGrid) {
auto chunks = MakeGrid(2, 2);
std::swap(chunks[0], chunks[3]);
Raw::Raw raw;
ASSERT_TRUE(Read(WriteRaw(chunks, 32, 2, 2), raw));
ASSERT_EQ(raw.chunkGrid.size(), 4);
EXPECT_EQ(raw.chunkGrid[0], 3);
EXPECT_EQ(raw.chunkGrid[3], 0);
EXPECT_EQ(raw.GetSceneIDAt(-7.0f, -7.0f), 1 * 4 + 1); // chunk id 0, local (1, 1)
}
TEST_F(RawTest, ChunksThatAreNotAGridAreSearchedOneByOne) {
auto chunks = MakeGrid(2, 1);
chunks[1].offsetZ = 3.0f; // shifted off the grid
Raw::Raw raw;
ASSERT_TRUE(Read(WriteRaw(chunks, 32, 2, 1), raw));
EXPECT_TRUE(raw.chunkGrid.empty());
EXPECT_EQ(raw.GetSceneIDAt(-7.0f, -7.0f), 1 * 4 + 1);
EXPECT_EQ(raw.GetSceneIDAt(1.0f, 4.0f), 16 + 1 * 4 + 1); // chunk 1, local (1, 1)
}
TEST_F(RawTest, ReadsVersion31SceneMapWithoutTheLastRowAndColumn) {
TestChunk chunk;
chunk.width = 3;
chunk.height = 3;
// resolution is width - 1 = 2; the file has a 3x3 scene map
chunk.sceneMap = { 1, 2, 9, 3, 4, 9, 9, 9, 9 };
const auto data = WriteRaw({ chunk }, 31);
std::istringstream stream(data);
Raw::Raw raw;
ASSERT_TRUE(Raw::ReadRaw(stream, raw));
EXPECT_EQ(stream.tellg(), static_cast<std::streamoff>(data.size()));
ASSERT_EQ(raw.chunks.size(), 1);
EXPECT_EQ(raw.chunks[0].colorMapResolution, 2);
EXPECT_EQ(raw.chunks[0].sceneMap, (std::vector<uint8_t>{ 1, 2, 3, 4 }));
EXPECT_EQ(raw.chunks[0].sceneMapLegacy, chunk.sceneMap);
EXPECT_EQ(raw.chunks[0].shaderId, 77);
EXPECT_EQ(raw.chunks[0].textureIds, (std::vector<uint32_t>{ 100, 101, 102, 103 }));
EXPECT_EQ(raw.chunks[0].colorMap.size(), 3 * 3 * 4); // width^2 pixels
EXPECT_TRUE(raw.chunks[0].lightMap.empty());
EXPECT_EQ(raw.chunks[0].textureMap.size(), 2 * 2 * 4);
EXPECT_EQ(raw.chunks[0].flairs.size(), 2);
EXPECT_TRUE(raw.chunks[0].meshTri.empty());
}
TEST_F(RawTest, Version30SceneMapIsReadButEveryCellIsSceneZero) {
TestChunk chunk;
chunk.width = 3;
chunk.height = 3;
// resolution is width - 1 = 2; the file has a 3x3 scene map the client doesn't use
chunk.sceneMap = { 7, 7, 7, 7, 7, 7, 7, 7, 7 };
const auto data = WriteRaw({ chunk, chunk }, 30, 1, 2);
std::istringstream stream(data);
Raw::Raw raw;
ASSERT_TRUE(Raw::ReadRaw(stream, raw));
EXPECT_EQ(stream.tellg(), static_cast<std::streamoff>(data.size()));
ASSERT_EQ(raw.chunks.size(), 2);
EXPECT_EQ(raw.chunks[1].sceneMapLegacy, chunk.sceneMap);
EXPECT_EQ(raw.chunks[1].sceneMap, (std::vector<uint8_t>{ 0, 0, 0, 0 }));
EXPECT_TRUE(raw.chunks[1].IsValidForSceneLookup());
EXPECT_EQ(raw.GetSceneIDAt(0.5f, 0.5f), 0);
}
TEST_F(RawTest, DevFileHasNoChunks) {
Raw::Raw raw;
ASSERT_TRUE(Read(WriteRaw({}, 32, 0, 0, 1), raw));
EXPECT_EQ(raw.dev, 1);
EXPECT_TRUE(raw.chunks.empty());
EXPECT_EQ(raw.GetSceneIDAt(0.0f, 0.0f), std::nullopt);
}
TEST_F(RawTest, RejectsBadFiles) {
Raw::Raw raw;
EXPECT_FALSE(Read("", raw));
const auto data = WriteRaw(MakeGrid(1, 2), 32, 1, 2);
EXPECT_FALSE(Read(data.substr(0, data.size() - 3), raw)); // truncated in the last chunk's mesh
std::string tooManyChunks = data;
const uint32_t chunkCount = 5000;
tooManyChunks.replace(3, sizeof(chunkCount), reinterpret_cast<const char*>(&chunkCount), sizeof(chunkCount));
EXPECT_FALSE(Read(tooManyChunks, raw));
// a 3x3 chunk's color map resolution is at 15 (header) + 40 (chunk info) + 36 (heights)
std::string hugeColorMap = WriteRaw({ TestChunk{} });
const uint32_t resolution = 5000;
hugeColorMap.replace(91, sizeof(resolution), reinterpret_cast<const char*>(&resolution), sizeof(resolution));
EXPECT_FALSE(Read(hugeColorMap, raw));
}
TEST_F(RawTest, SceneCellIsTheNearestOneLikeTheClient) {
Raw::Raw raw;
ASSERT_TRUE(Read(WriteRaw(MakeGrid(1, 1, 0.0f, 0.0f)), raw));
const auto& chunk = raw.chunks[0];
// cells are 2 units; floor(position / 2 + 0.5), clamped to [0, 3]; index [x cell * 4 + z cell]
EXPECT_EQ(chunk.GetSceneIDAtLocal(0.0f, 0.0f), 0);
EXPECT_EQ(chunk.GetSceneIDAtLocal(0.9f, 0.0f), 0);
EXPECT_EQ(chunk.GetSceneIDAtLocal(1.0f, 0.0f), 4);
EXPECT_EQ(chunk.GetSceneIDAtLocal(0.0f, 3.1f), 2);
EXPECT_EQ(chunk.GetSceneIDAtLocal(5.0f, 1.0f), 13);
EXPECT_EQ(chunk.GetSceneIDAtLocal(100.0f, 100.0f), 15);
EXPECT_EQ(chunk.GetSceneIDAtLocal(-100.0f, -100.0f), 0);
// grid points are 2 units apart: point (i, j) is at (2i, 2j)
EXPECT_EQ(chunk.GetSceneIDAtGrid(1, 2), 1 * 4 + 2);
EXPECT_EQ(chunk.GetSceneIDAtGrid(4, 4), 15);
}
TEST_F(RawTest, SceneAtPositionMatchesTheClient) {
Raw::Raw raw;
ASSERT_TRUE(Read(WriteRaw(MakeGrid(2, 2), 32, 2, 2), raw));
// inside chunk 0 ([-8, 0] x [-8, 0])
EXPECT_EQ(raw.GetSceneIDAt(-8.0f, -8.0f), 0);
EXPECT_EQ(raw.GetSceneIDAt(-5.0f, -8.0f), 4 * 2);
// chunk 3 ([0, 8] x [0, 8])
EXPECT_EQ(raw.GetSceneIDAt(0.5f, 0.5f), 48);
// within 0.001 of a chunk's start counts as the next chunk
EXPECT_EQ(raw.GetSceneIDAt(-0.0005f, -7.0f), 32 + 1);
EXPECT_EQ(raw.GetSceneIDAt(-0.01f, -7.0f), 3 * 4 + 1);
// outside the terrain clamps to its edge
EXPECT_EQ(raw.GetSceneIDAt(-500.0f, -500.0f), 0);
// the far edge is kept one scene cell inside, so it is the last cell of the last chunk
EXPECT_EQ(raw.GetSceneIDAt(8.0f, 8.0f), 48 + 15);
EXPECT_EQ(raw.GetSceneIDAt(500.0f, 500.0f), 48 + 15);
}
TEST_F(RawTest, GridLookupMatchesCheckingEveryChunk) {
auto chunks = MakeGrid(3, 4, -1126.4f, -921.6f);
for (auto& chunk : chunks) {
chunk.scale = 3.2f;
chunk.offsetX = -1126.4f + (chunk.offsetX + 1126.4f) / 8.0f * 12.8f;
chunk.offsetZ = -921.6f + (chunk.offsetZ + 921.6f) / 8.0f * 12.8f;
}
chunks.erase(chunks.begin() + 5); // a hole in the grid
Raw::Raw raw;
ASSERT_TRUE(Read(WriteRaw(chunks, 32, 3, 4), raw));
ASSERT_FALSE(raw.chunkGrid.empty());
Raw::Raw linear = raw;
linear.chunkGrid.clear();
std::mt19937 random(1);
std::uniform_real_distribution<float> randomX(raw.minBoundsX - 20.0f, raw.maxBoundsX + 20.0f);
std::uniform_real_distribution<float> randomZ(raw.minBoundsZ - 20.0f, raw.maxBoundsZ + 20.0f);
for (int i = 0; i < 20000; ++i) {
float x = randomX(random);
float z = randomZ(random);
// every few positions on a chunk edge
if (i % 4 == 0) x = raw.minBoundsX + std::round((x - raw.minBoundsX) / raw.chunkSpanX) * raw.chunkSpanX;
if (i % 5 == 0) z = raw.minBoundsZ + std::round((z - raw.minBoundsZ) / raw.chunkSpanZ) * raw.chunkSpanZ;
ASSERT_EQ(raw.GetSceneIDAt(x, z), linear.GetSceneIDAt(x, z)) << x << ", " << z;
}
}
TEST_F(RawTest, GridToWorldPosition) {
Raw::Raw raw;
ASSERT_TRUE(Read(WriteRaw(MakeGrid(1, 1)), raw));
const auto position = raw.chunks[0].GridToWorldPos(1, 3);
EXPECT_FLOAT_EQ(position.x, -8.0f + 2.0f);
EXPECT_FLOAT_EQ(position.y, 1 * 5 + 3);
EXPECT_FLOAT_EQ(position.z, -8.0f + 6.0f);
}
TEST_F(RawTest, TerrainMeshForNonSquareChunks) {
TestChunk first;
first.width = 3;
first.height = 4;
first.colorMapResolution = 2;
first.sceneMap = { 1, 2, 3, 4 };
TestChunk second = first;
second.offsetX = 2.0f;
Raw::Raw raw;
ASSERT_TRUE(Read(WriteRaw({ first, second }, 32, 2, 1), raw));
Raw::TerrainMesh mesh;
Raw::GenerateTerrainMesh(raw, mesh);
ASSERT_EQ(mesh.vertices.size(), 2 * 12);
ASSERT_EQ(mesh.triangles.size(), 2 * (2 * 3) * 2 * 3);
for (const auto index : mesh.triangles) EXPECT_LT(index, mesh.vertices.size());
// vertex [i * height + j]; the first quad of the first chunk
EXPECT_EQ(mesh.triangles[0], 1 * 4 + 1);
EXPECT_EQ(mesh.triangles[1], 1 * 4 + 0);
EXPECT_EQ(mesh.triangles[2], 0);
EXPECT_FLOAT_EQ(mesh.vertices[1 * 4 + 3].position.x, 1.0f);
EXPECT_FLOAT_EQ(mesh.vertices[1 * 4 + 3].position.z, 3.0f);
EXPECT_EQ(mesh.vertices[2 * 4 + 3].sceneID, 4);
// the second chunk's vertices come after the first's
EXPECT_FLOAT_EQ(mesh.vertices[12].position.x, 2.0f);
EXPECT_EQ(mesh.triangles[36], 12 + 1 * 4 + 1);
}
TEST_F(RawTest, NoTerrainMeshWithoutSceneData) {
TestChunk chunk;
Raw::Raw raw;
ASSERT_TRUE(Read(WriteRaw({ chunk }, 30), raw));
Raw::TerrainMesh mesh;
Raw::GenerateTerrainMesh(raw, mesh);
EXPECT_TRUE(mesh.vertices.empty());
EXPECT_TRUE(mesh.triangles.empty());
}
TEST_F(RawTest, WritesTheTerrainMeshAsObj) {
Raw::TerrainMesh mesh;
mesh.vertices = { Raw::SceneVertex(NiPoint3(0.0f, 1.0f, 2.0f), 1), Raw::SceneVertex(NiPoint3(3.0f, 4.0f, 5.0f), 200), Raw::SceneVertex() };
mesh.triangles = { 0, 1, 2 };
const auto path = (std::filesystem::temp_directory_path() / "dlu_raw_test.obj").string();
ASSERT_TRUE(Raw::WriteTerrainMeshToOBJ(mesh, path));
std::vector<std::string> lines;
{
std::ifstream file(path);
for (std::string line; std::getline(file, line);) lines.push_back(line);
} // closed before removing it (Windows can't remove an open file)
std::filesystem::remove(path);
ASSERT_EQ(lines.size(), 4);
EXPECT_EQ(lines[0], "v 0 1 2 1 0 0"); // scene 1 is red
EXPECT_EQ(lines[1], "v 3 4 5 1 1 1"); // past the table: white
EXPECT_EQ(lines[3], "f 1 2 3");
EXPECT_FALSE(Raw::WriteTerrainMeshToOBJ(mesh, "/nonexistent-directory/terrain.obj"));
}
TEST(SceneColorTest, TemplateColorsAndFallback) {
EXPECT_FLOAT_EQ(SceneColor::Get(0).m_Red, 128.0f / 255.0f);
EXPECT_FLOAT_EQ(SceneColor::Get(3).m_Blue, 1.0f);
EXPECT_FLOAT_EQ(SceneColor::Get(145).m_Green, SceneColor::TEMPLATE_COLORS.back().m_Green);
EXPECT_EQ(&SceneColor::Get(146), &SceneColor::FALLBACK_COLOR);
EXPECT_EQ(&SceneColor::Get(255), &SceneColor::FALLBACK_COLOR);
}
TEST(NiColorTest, ConstructionAndLuminance) {
constexpr NiColor black;
EXPECT_FLOAT_EQ(black.m_Red, 0.0f);
EXPECT_FLOAT_EQ(black.Luminance(), 0.0f);
EXPECT_NEAR(NiColor(1.0f, 1.0f, 1.0f).Luminance(), 1.0f, 0.0001f);
EXPECT_FLOAT_EQ(NiColor(0.0f, 1.0f, 0.0f).Luminance(), 0.71516f);
}

View File

@@ -0,0 +1,193 @@
#pragma once
#include <cstdint>
#include <sstream>
#include <string>
#include <vector>
#include "NiPoint3.h"
// Writes .raw terrain and .luz zone files in the client's layout, for the zone manager tests
namespace TerrainTestWriter {
template<typename T>
void Write(std::ostream& stream, const T& value) {
stream.write(reinterpret_cast<const char*>(&value), sizeof(T));
}
inline void WriteBytes(std::ostream& stream, size_t count, uint8_t value = 0xAB) {
const std::string bytes(count, static_cast<char>(value));
stream.write(bytes.data(), bytes.size());
}
inline void WriteString8(std::ostream& stream, const std::string& value) {
Write<uint8_t>(stream, static_cast<uint8_t>(value.size()));
stream.write(value.data(), value.size());
}
struct TestChunk {
uint32_t id = 0;
uint32_t width = 3;
uint32_t height = 3;
float offsetX = 0.0f;
float offsetZ = 0.0f;
float scale = 1.0f;
// Version 32+: the color map resolution; older versions use width - 1
uint32_t colorMapResolution = 2;
// colorMapResolution^2 scene IDs ((resolution + 1)^2 for version 31 and older); empty = all 0
std::vector<uint8_t> sceneMap;
// width * height heights; empty = the point index
std::vector<float> heights;
uint32_t lightMapSize = 5;
uint32_t textureMapResolution = 2;
uint32_t blendMapSize = 7;
uint32_t flairCount = 2;
uint32_t vertSize = 4;
std::vector<uint16_t> triangleListSizes = { 3, 0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3 };
};
inline void WriteChunk(std::ostream& stream, const TestChunk& chunk, uint16_t version) {
Write(stream, chunk.id);
Write(stream, chunk.width);
Write(stream, chunk.height);
Write(stream, chunk.offsetX);
Write(stream, chunk.offsetZ);
if (version < 32) Write<uint32_t>(stream, 77); // shader ID
for (uint32_t i = 0; i < 4; ++i) Write<uint32_t>(stream, 100 + i); // texture IDs
Write(stream, chunk.scale);
for (uint32_t i = 0; i < chunk.width * chunk.height; ++i) {
Write<float>(stream, chunk.heights.empty() ? static_cast<float>(i) : chunk.heights[i]);
}
const uint32_t resolution = version >= 32 ? chunk.colorMapResolution : chunk.width - 1;
if (version >= 32) {
Write(stream, resolution);
WriteBytes(stream, static_cast<size_t>(resolution) * resolution * 4);
Write(stream, chunk.lightMapSize);
WriteBytes(stream, chunk.lightMapSize);
} else {
WriteBytes(stream, static_cast<size_t>(chunk.width) * chunk.width * 4);
}
Write(stream, chunk.textureMapResolution);
WriteBytes(stream, static_cast<size_t>(chunk.textureMapResolution) * chunk.textureMapResolution * 4);
if (version >= 32) {
Write<uint8_t>(stream, 1); // texture settings
Write(stream, chunk.blendMapSize);
WriteBytes(stream, chunk.blendMapSize);
}
// Flair: id, scale, position, rotation, RGB, padding byte (36 bytes)
Write(stream, chunk.flairCount);
for (uint32_t i = 0; i < chunk.flairCount; ++i) {
Write<uint32_t>(stream, 500 + i);
Write<float>(stream, 1.5f);
Write(stream, NiPoint3(1.0f + i, 2.0f, 3.0f));
Write(stream, NiPoint3(0.0f, 90.0f, 0.0f));
for (const uint8_t channel : { 10, 20, 30, 40 }) Write(stream, channel);
}
// version 31 and older: one more row and column
const size_t cells = version >= 32 ? static_cast<size_t>(resolution) * resolution : static_cast<size_t>(resolution + 1) * (resolution + 1);
for (size_t i = 0; i < cells; ++i) Write<uint8_t>(stream, chunk.sceneMap.empty() ? 0 : chunk.sceneMap[i]);
if (version < 32) return;
Write(stream, chunk.vertSize);
if (chunk.vertSize == 0) return;
WriteBytes(stream, static_cast<size_t>(chunk.vertSize) * sizeof(uint16_t));
WriteBytes(stream, 16 * sizeof(uint16_t));
for (uint32_t i = 0; i < 16; ++i) {
const uint16_t size = i < chunk.triangleListSizes.size() ? chunk.triangleListSizes[i] : 0;
Write(stream, size);
WriteBytes(stream, static_cast<size_t>(size) * sizeof(uint16_t));
}
}
inline std::string WriteRaw(const std::vector<TestChunk>& chunks, uint16_t version = 32, uint32_t chunksX = 1, uint32_t chunksZ = 1, uint8_t dev = 0) {
std::ostringstream stream;
Write(stream, version);
Write(stream, dev);
if (dev == 0) {
Write<uint32_t>(stream, static_cast<uint32_t>(chunks.size()));
Write(stream, chunksZ); // the header's "width" is the chunk count along Z
Write(stream, chunksX);
for (const auto& chunk : chunks) WriteChunk(stream, chunk, version);
}
return stream.str();
}
struct TestScene {
std::string filename;
uint32_t id = 0;
uint32_t layer = 0;
std::string name;
uint8_t r = 0, g = 0, b = 0;
};
struct TestTransitionPoint {
uint32_t sceneID = 0;
uint32_t layerID = 0;
NiPoint3 position;
};
// A .luz (version 36 to 41) with no paths
inline std::string WriteLuz(uint32_t worldID, const std::vector<TestScene>& scenes, uint8_t boundaryCount, const std::string& rawName,
const std::vector<std::pair<TestTransitionPoint, TestTransitionPoint>>& transitions, uint32_t version = 41) {
std::ostringstream stream;
Write(stream, version);
Write<uint32_t>(stream, 5); // revision
Write(stream, worldID);
if (version >= 38) {
Write(stream, NiPoint3(1.0f, 2.0f, 3.0f)); // spawn position
for (const float value : { 1.0f, 0.0f, 0.0f, 0.0f }) Write(stream, value); // spawn rotation
}
// the scene count is a byte before version 37
if (version >= 37) Write<uint32_t>(stream, static_cast<uint32_t>(scenes.size()));
else Write<uint8_t>(stream, static_cast<uint8_t>(scenes.size()));
for (const auto& scene : scenes) {
WriteString8(stream, scene.filename);
Write(stream, scene.id);
Write(stream, scene.layer);
WriteString8(stream, scene.name);
Write(stream, scene.r);
Write(stream, scene.b);
Write(stream, scene.g);
}
// Zone boundary lines: normal, point, destination zone, destination scene, spawn location (44 bytes)
Write(stream, boundaryCount);
for (uint8_t i = 0; i < boundaryCount; ++i) {
Write(stream, NiPoint3(0.0f, 0.0f, 1.0f));
Write(stream, NiPoint3(10.0f, 0.0f, static_cast<float>(i)));
Write<uint16_t>(stream, 1100); // map
Write<uint16_t>(stream, i + 1); // instance
Write<uint32_t>(stream, 2);
Write(stream, NiPoint3(5.0f, 6.0f, 7.0f));
}
WriteString8(stream, rawName);
WriteString8(stream, "Test Zone");
WriteString8(stream, "A zone for tests");
Write<uint32_t>(stream, static_cast<uint32_t>(transitions.size()));
for (const auto& [from, to] : transitions) {
// before version 40 a transition has a name and a width
if (version < 40) {
WriteString8(stream, "transition");
Write(stream, 2.5f);
}
// five points from version 34 to 38 (the last three repeat the second here), two otherwise
const size_t pointCount = version >= 34 && version < 39 ? 5 : 2;
for (size_t i = 0; i < pointCount; ++i) {
const auto& point = i == 0 ? from : to;
Write(stream, point.sceneID);
Write(stream, point.layerID);
Write(stream, point.position);
}
}
Write<uint32_t>(stream, 8); // path data length
Write<uint32_t>(stream, 1); // path chunk version
Write<uint32_t>(stream, 0); // path count
return stream.str();
}
}

View File

@@ -0,0 +1,243 @@
#include "GameDependencies.h"
#include <algorithm>
#include <filesystem>
#include <fstream>
#include "AssetManager.h"
#include "CDZoneTableTable.h"
#include "DEVGMCommands.h"
#include "Entity.h"
#include "TerrainTestWriter.h"
#include "Zone.h"
#include "dZoneManager.h"
using namespace TerrainTestWriter;
// The zone table's storage (CDZoneTableTable.cpp), to add the test zone
namespace CDZoneTableTable {
extern Table entries;
}
// Loads a written test zone (a .luz and its .raw) through the zone manager
class ZoneSceneTest : public GameDependenciesTest {
protected:
static constexpr LWOMAPID TEST_MAP = 9990;
std::filesystem::path root;
void SetUp() override {
SetUpDependencies();
root = std::filesystem::temp_directory_path() / (std::string("dlu_zone_scene_test_") + ::testing::UnitTest::GetInstance()->current_test_info()->name());
std::filesystem::create_directories(root / "res" / "maps" / "testzone");
std::ofstream(root / "res" / "CDServer.sqlite").close(); // marks an unpacked client
Game::assetManager = new AssetManager(root);
CDZoneTable zone{};
zone.zoneID = TEST_MAP;
zone.zoneName = "testzone/test.luz";
CDZoneTableTable::entries[TEST_MAP] = zone;
}
void TearDown() override {
CDZoneTableTable::entries.erase(TEST_MAP);
delete Game::assetManager;
Game::assetManager = nullptr;
std::filesystem::remove_all(root);
TearDownDependencies();
}
void WriteFile(const std::string& name, const std::string& data) const {
std::ofstream file(root / "res" / "maps" / "testzone" / name, std::ios::binary);
file.write(data.data(), data.size());
}
// Scenes 0 (global), 1, 2 and 3 (general) and 5 (audio, layer 1); transitions 1 <-> 2 and 2 <-> 3
void WriteZone(const std::string& rawName = "test.raw", uint8_t boundaryCount = 2, uint32_t version = 41) const {
const std::vector<TestScene> scenes = {
{ "global.lvl", 0, 0, "Global", 10, 20, 30 },
{ "one.lvl", 1, 0, "Scene One", 255, 0, 0 },
{ "two.lvl", 2, 0, "Scene Two", 0, 255, 0 },
{ "three.lvl", 3, 0, "Scene Three", 0, 0, 255 },
{ "audio.lvl", 5, 1, "Audio", 0, 0, 0 },
};
WriteFile("test.luz", WriteLuz(TEST_MAP, scenes, boundaryCount, rawName, {
{ { 1, 0, NiPoint3(1.0f, 0.0f, 0.0f) }, { 2, 0, NiPoint3(2.0f, 0.0f, 0.0f) } },
{ { 2, 0, NiPoint3(3.0f, 0.0f, 0.0f) }, { 3, 0, NiPoint3(4.0f, 0.0f, 0.0f) } },
}, version));
}
// One 3x3 point chunk from (0, 0) to (4, 4) whose 2x2 scene map holds scenes 1 2 / 3 1
static std::string MakeRaw() {
TestChunk chunk;
chunk.width = 3;
chunk.height = 3;
chunk.scale = 2.0f;
chunk.colorMapResolution = 2;
chunk.sceneMap = { 1, 2, 3, 1 };
return WriteRaw({ chunk });
}
Zone* LoadZone() const {
Game::zoneManager->LoadZone(LWOZONEID(TEST_MAP, 0, 0));
Zone* zone = Game::zoneManager->GetZoneMut();
zone->LoadZoneIntoMemory(); // the levels the scenes name aren't written
Game::zoneManager->BuildSceneGraph();
return zone;
}
static bool Contains(const std::vector<LWOSCENEID>& scenes, LWOSCENEID scene) {
return std::ranges::find(scenes, scene) != scenes.end();
}
};
TEST_F(ZoneSceneTest, ReadsScenesBoundariesAndTransitions) {
WriteZone();
WriteFile("test.raw", MakeRaw());
const Zone* zone = LoadZone();
EXPECT_EQ(zone->GetWorldID(), TEST_MAP);
EXPECT_EQ(zone->GetZoneRawPath(), "test.raw");
EXPECT_EQ(zone->GetZoneName(), "Test Zone"); // read after the boundary lines
EXPECT_EQ(zone->GetZonePath(), "maps/testzone/");
const auto& boundaries = zone->GetBoundaries();
ASSERT_EQ(boundaries.size(), 2);
EXPECT_FLOAT_EQ(boundaries[1].normal.z, 1.0f);
EXPECT_FLOAT_EQ(boundaries[1].point.x, 10.0f);
EXPECT_FLOAT_EQ(boundaries[1].point.z, 1.0f);
EXPECT_EQ(boundaries[1].destinationZone.GetMapID(), 1100);
EXPECT_EQ(boundaries[1].destinationZone.GetInstanceID(), 2);
EXPECT_EQ(boundaries[1].destinationZone.GetCloneID(), 0);
EXPECT_EQ(boundaries[1].destinationSceneID, 2);
EXPECT_FLOAT_EQ(boundaries[1].spawnLocation.y, 6.0f);
ASSERT_EQ(zone->GetScenes().size(), 5);
const SceneRef* one = zone->GetScene(LWOSCENEID(1, 0));
ASSERT_NE(one, nullptr);
EXPECT_EQ(one->name, "Scene One");
EXPECT_EQ(one->sceneType, eSceneType::General);
EXPECT_FLOAT_EQ(one->color.m_Red, 1.0f);
const SceneRef* global = zone->GetScene(LWOSCENEID(0, 0));
ASSERT_NE(global, nullptr);
// stored in the file as red, blue, green
EXPECT_FLOAT_EQ(global->color.m_Red, 10 / 255.0f);
EXPECT_FLOAT_EQ(global->color.m_Green, 20 / 255.0f);
EXPECT_FLOAT_EQ(global->color.m_Blue, 30 / 255.0f);
const SceneRef* audio = zone->GetScene(LWOSCENEID(5, 1));
ASSERT_NE(audio, nullptr);
EXPECT_EQ(audio->sceneType, eSceneType::Audio);
EXPECT_EQ(zone->GetScene(LWOSCENEID(4, 0)), nullptr);
const auto& transitions = zone->GetSceneTransitions();
ASSERT_EQ(transitions.size(), 2);
ASSERT_EQ(transitions[1].points.size(), 2);
EXPECT_EQ(transitions[1].points[0].sceneID, LWOSCENEID(2, 0));
EXPECT_EQ(transitions[1].points[1].sceneID, LWOSCENEID(3, 0));
EXPECT_FLOAT_EQ(transitions[1].points[1].position.x, 4.0f);
}
TEST_F(ZoneSceneTest, ZoneWithoutBoundaryLines) {
WriteZone("test.raw", 0);
WriteFile("test.raw", MakeRaw());
const Zone* zone = LoadZone();
EXPECT_EQ(zone->GetZoneName(), "Test Zone");
EXPECT_TRUE(zone->GetBoundaries().empty());
EXPECT_EQ(zone->GetSceneTransitions().size(), 2);
}
// Version 37 has a u32 scene count (a byte before it) and no spawn point; 36 and 37 have named five point transitions
TEST_F(ZoneSceneTest, ReadsOlderZoneVersions) {
for (const uint32_t version : { 36u, 37u, 38u, 39u, 40u }) {
WriteZone("test.raw", 2, version);
WriteFile("test.raw", MakeRaw());
const Zone* zone = LoadZone();
EXPECT_EQ(zone->GetScenes().size(), 5) << version;
EXPECT_EQ(zone->GetBoundaries().size(), 2) << version;
EXPECT_EQ(zone->GetZoneRawPath(), "test.raw") << version;
EXPECT_EQ(zone->GetZoneName(), "Test Zone") << version;
ASSERT_EQ(zone->GetSceneTransitions().size(), 2) << version;
const auto& transition = zone->GetSceneTransitions()[1];
EXPECT_EQ(transition.points.size(), version < 39 ? 5 : 2) << version;
EXPECT_EQ(transition.points[1].sceneID, LWOSCENEID(3, 0)) << version;
if (version < 40) {
EXPECT_EQ(transition.name, "transition") << version;
EXPECT_FLOAT_EQ(transition.width, 2.5f) << version;
}
EXPECT_FALSE(zone->GetZoneRaw().chunks.empty()) << version;
}
}
TEST_F(ZoneSceneTest, SceneAtPosition) {
WriteZone();
WriteFile("test.raw", MakeRaw());
LoadZone();
EXPECT_EQ(Game::zoneManager->GetSceneIDFromPosition(NiPoint3(0.5f, 100.0f, 0.5f)), LWOSCENEID(1, 0));
EXPECT_EQ(Game::zoneManager->GetSceneIDFromPosition(NiPoint3(0.5f, -50.0f, 3.5f)), LWOSCENEID(2, 0));
EXPECT_EQ(Game::zoneManager->GetSceneIDFromPosition(NiPoint3(3.5f, 0.0f, 0.5f)), LWOSCENEID(3, 0));
EXPECT_EQ(Game::zoneManager->GetSceneIDFromPosition(NiPoint3(50.0f, 0.0f, 50.0f)), LWOSCENEID(1, 0));
}
TEST_F(ZoneSceneTest, SceneGraphFromTransitions) {
WriteZone();
WriteFile("test.raw", MakeRaw());
LoadZone();
const auto& graph = Game::zoneManager->GetSceneAdjacencyList();
EXPECT_EQ(graph.size(), 4); // the audio scene is left out
EXPECT_FALSE(graph.contains(LWOSCENEID(5, 1)));
const auto one = Game::zoneManager->GetAdjacentScenes(LWOSCENEID(1, 0));
EXPECT_EQ(one.size(), 2);
EXPECT_TRUE(Contains(one, LWOSCENEID(2, 0)));
EXPECT_TRUE(Contains(one, LWOSCENEID(0, 0)));
const auto two = Game::zoneManager->GetAdjacentScenes(LWOSCENEID(2, 0));
EXPECT_EQ(two.size(), 3);
EXPECT_TRUE(Contains(two, LWOSCENEID(1, 0)));
EXPECT_TRUE(Contains(two, LWOSCENEID(3, 0)));
// the global scene is next to every other scene
const auto global = Game::zoneManager->GetAdjacentScenes(LWOSCENEID(0, 0));
EXPECT_EQ(global.size(), 3);
EXPECT_FALSE(Contains(global, LWOSCENEID(0, 0)));
EXPECT_TRUE(Game::zoneManager->GetAdjacentScenes(LWOSCENEID(9, 0)).empty());
// building again doesn't add anything twice
Game::zoneManager->BuildSceneGraph();
EXPECT_EQ(Game::zoneManager->GetAdjacentScenes(LWOSCENEID(2, 0)).size(), 3);
}
TEST_F(ZoneSceneTest, MissingRawStillLoadsTheZone) {
WriteZone("missing.raw");
const Zone* zone = LoadZone();
EXPECT_EQ(zone->GetZoneName(), "Test Zone");
EXPECT_EQ(zone->GetSceneTransitions().size(), 2);
EXPECT_TRUE(zone->GetZoneRaw().chunks.empty());
EXPECT_EQ(Game::zoneManager->GetSceneIDFromPosition(NiPoint3(0.5f, 0.0f, 0.5f)), LWOSCENEID_INVALID);
}
TEST_F(ZoneSceneTest, UnreadableRawStillLoadsTheZone) {
WriteZone();
const auto raw = MakeRaw();
WriteFile("test.raw", raw.substr(0, raw.size() / 2));
const Zone* zone = LoadZone();
EXPECT_EQ(zone->GetZoneName(), "Test Zone");
EXPECT_TRUE(zone->GetZoneRaw().chunks.empty());
EXPECT_EQ(Game::zoneManager->GetSceneIDFromPosition(NiPoint3(0.5f, 0.0f, 0.5f)), LWOSCENEID_INVALID);
}
TEST_F(ZoneSceneTest, SceneCommandsReport) {
WriteZone();
WriteFile("test.raw", MakeRaw());
LoadZone();
EntityInfo playerInfo = info;
playerInfo.pos = NiPoint3(0.5f, 0.0f, 0.5f); // scene 1
auto player = std::make_unique<Entity>(1, playerInfo);
// the scene commands only report; they must not fail
DEVGMCommands::GetScene(player.get(), UNASSIGNED_SYSTEM_ADDRESS, "");
DEVGMCommands::GetAdjacentScenes(player.get(), UNASSIGNED_SYSTEM_ADDRESS, "");
}