feat(dCommon): read cdclient.fdb in place through its hash buckets

FdbMappedFile maps a file read-only (CreateFileMapping/MapViewOfFile on
Windows, mmap elsewhere) and falls back to reading it into memory when
mapping fails. FdbReader reads the table and column headers from it and
looks rows up by their first column through the fdb's own buckets,
decoding every integer as little-endian with bounds checks, so the rows
never get copied out of the file.

Tests write small fdb files (collisions, text, int64, nulls) and read
them both mapped and from memory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Aaron Kimbrell
2026-09-29 21:56:18 -05:00
parent 94b411a32e
commit d670484b68
8 changed files with 1020 additions and 0 deletions

58
dCommon/FdbMappedFile.h Normal file
View File

@@ -0,0 +1,58 @@
#ifndef FDBMAPPEDFILE_H
#define FDBMAPPEDFILE_H
#include <cstdint>
#include <filesystem>
#include <vector>
/**
* A read-only view of a whole file. The file is memory-mapped when the platform allows it
* (CreateFileMapping/MapViewOfFile on Windows, mmap elsewhere), so every process that maps the
* same file shares one copy through the OS page cache. If mapping fails the file is read into
* memory instead, and the view works the same way.
*/
class FdbMappedFile {
public:
FdbMappedFile() = default;
~FdbMappedFile();
FdbMappedFile(const FdbMappedFile&) = delete;
FdbMappedFile& operator=(const FdbMappedFile&) = delete;
FdbMappedFile(FdbMappedFile&& other) noexcept;
FdbMappedFile& operator=(FdbMappedFile&& other) noexcept;
/**
* Opens a file. Closes whatever was open before.
*
* @param path The file to open
* @param allowMapping false skips the mapping and reads the file into memory
* @return true if the file is open and not empty
*/
bool Open(const std::filesystem::path& path, bool allowMapping = true);
void Close();
[[nodiscard]] bool IsOpen() const { return m_Data != nullptr; }
// true if the view is a mapping, false if the file was read into memory
[[nodiscard]] bool IsMapped() const { return m_Mapped; }
[[nodiscard]] const uint8_t* GetData() const { return m_Data; }
[[nodiscard]] uint64_t GetSize() const { return m_Size; }
private:
bool Map(const std::filesystem::path& path);
bool Read(const std::filesystem::path& path);
const uint8_t* m_Data = nullptr;
uint64_t m_Size = 0;
bool m_Mapped = false;
std::vector<uint8_t> m_Buffer;
#ifdef _WIN32
void* m_FileHandle = nullptr;
void* m_MappingHandle = nullptr;
#endif
};
#endif // FDBMAPPEDFILE_H