GitHub - NVIDIAGameWorks/Displacement-MicroMap-Toolkit (original) (raw)

Important

This project has been archived and is no longer maintained. The vulkan extension VK_NV_displacement_micromap is no longer available.

We recommend exploring NVIDIA RTX Mega Geometry, which can provide similar functionality with greater flexibility. Seevk_tessellated_clusters, which demonstrates raytracing displacement with Vulkan.

NVIDIA Displacement Micro-Map Toolkit

This toolkit provides libraries, samples and tools to create and view displaced micromeshes. It is a work in progress and feedback is welcome!

The latest driver exposing VK_NV_displacement_micromap is required to raytrace micromeshes, such as the Vulkan Beta Driver available athttps://developer.nvidia.com/vulkan-driver.

We recommend to check the Micro-Mesh Basics slidesfirst as well as the dmm_displacementmini-sample.

There are also NVIDIA GTC presentations Getting Started with Compressed Micro-Meshes [S51410]and Interactive GPU-Based Remeshing of Large Meshes [S51567].

The NVIDIA Micro-Mesh technology covers both opacity micromaps and displacement micromaps. This SDK currently solely covers displacements as a separate SDK exists for opacity.

Building

This repository contains submodules. After cloning, initialize them with:

git submodule update --init --recursive --jobs 8

The one manual dependency is the Vulkan SDK. For Windows, make sure to install the optional "glm" during the install or set the cmake GLM_INCLUDE_DIR location if installed separately.

Further dependencies will be downloaded during the cmake configuration step.

Windows

Tested with Visual Studio 2017, 2019 and 2022. Usecmake-gui to generate project files.

See docs/examples.md for usage examples.

Linux

Tested with gcc 11.2.1 and 11.3.0. This assumes build tools and cmake are already installed.

See docs/examples.md for usage examples.

Example Micromeshes

The toolkit provides some basic tools to process and bake displacement micromaps. Think of it like creating a heightmap to apply to a low-poly mesh, except the heightmap is highly compressed, supports hardware accelerated raytracing and doesn't need UVs. Some example starting assets are provided here to demonstrate toolkit usage. Download these to get started but feel free to try your own.

Note that these tools currently only support assets in glTF format. Everything else must be converted.

See docs/examples.md for typical asset processing examples using these meshes.

SDK and Toolkit Structure

micromesh SDK

The NVIDIA Displacement Micro-Map SDKis the low-level API meant for embedding in other applications and tools. It has a C-style API as well as an API agnostic GPU interface to facilitate this. As a result it is sometimes a bit less easy to use. All functionality is provided through the micromesh namespace and it makes frequent use of the micromesh::ArrayInfostructure, which allows it to pass data as a pointer & stride combination. All user visible data is allocated by the user, so some operations are executed in two steps where a micromeshOpSomethingBegin returns the sizing required, while micromeshOpSomethingEnd completes it. One can also abort such operations with micromeshOpContextAbort. The micromesh::Context therefore is stateful but fairly lightweight, in case you want to create one per thread. Right now there is also some rudimentary automatic threading within the context.

bary file

meshops API

The meshops API is a layer above the micromesh SDK, that provides easier usage and higher level operations, such as baking. The functions typically operate on a single mesh provided in memory through an abstraction called meshops::MeshView. This layer makes use of a Vulkan implementation for the various GPU-based operations and leverages the nvpro_core framework doing so.

This meshops layer is still a bit work in progress as we are migrating additional capabilities to it and might see a few more changes than the micromesh API.

tools & libraries

All tools operate on glTF 2.0 files and do support additional micromap specific glTF extensions from NVIDIA. The specifications are available atKhronosGroup/glTF#2273.

All micromap data is stored as .bary files which is a new container / file format that was specifically designed to allow direct storage of data consumed by the raytracing APIs without additional processing. See the open-sourceNVIDIA Displacement Micro-Map BaryFilefor more details.

mini samples

The mini-samples showcase our APIs in a minimal fashion, not relying on loading models or other files, but just procedurally generating content to reduce the amount of complexity

At the time of writing VK_NV_displacement_micromap is in beta still, meaning the extension is subject to changes. Do not use its headers here in production code!

micromesh toolbox

The micromesh_toolbox is a graphical workbench that allows inspecting micromeshes as well as interacting with some of the tools. It relies on VK_NV_mesh_shader to allow rasterized display of micromeshes. VK_KHR_acceleration_structure is required for baking micromaps. If available, VK_NV_displacement_micromap is used to render micromeshes with raytracing when choosing Rendering -> RTX. VK_NV_displacement_micromap was introduced with the RTX 40 Series Ada Lovelace architecture based GPUs. Previous RTX cards have support, but performance will be better with Ada. If you see a message about the missing extension, update to the latest driver (note: beta drivers are available at https://developer.nvidia.com/vulkan-driver).

The rasterization of micromeshes, especially compressed, is a bit of a more complex topic on its own. A dedicated sample is available athttps://github.com/nvpro-samples/vk_displacement_micromaps.

At the time of writing VK_NV_displacement_micromap is in beta still, meaning the extension is subject to changes. Do not use its headers here in production code!

micromesh tool

The command line tool, micromesh_tool, can be used for automation and defining persistent asset pipelines.

micromesh python

The python bindings for the toolkit, summarized in a Jupyter Notebook, can be used for integrating asset pipelines inside any application supporting a Python environment.

See micromesh_python for more details.

Fundamental Data Structures

A micromesh is the result of subdividing a base triangle using a power of two scheme. It is made of microtriangles and microvertices.

Triangle W,U,V

          V                     V                       V
          x                     x                       x
         / \                   / \ micro               / \
        /   \                 /   \ triangle          x _ x microvertex
       /     \               /     \                 / \ / \
      /       \             x _____ x               x _ x _ x
     /         \           / \     / \             / \ / \ / \
    /           \         /   \   /   \           x _ x _ x _ x
   /             \       /     \ /     \         / \ / \ / \ / \ 
  x _____________ x     x _____ x _____ x       x _ x _ x _ x _ x
W                  U   W                  U   W                   U

Subdivision level 0    Subdivision level 1    Subdivision level 2
  1 microtriangle        4 microtriangles       16 microtriangles
  aka base triangle

See docs/data_structures.md for more details on how this is used to generate displacement.

Micromesh Asset Pipeline

We describe the core steps that would be performed to create a micromesh asset from an original input mesh in the Micro-Mesh Asset Pipeline slide-deckas well as in docs/asset_pipeline.md

About the Latest Release

Version 1.2

Version 1.1

Third-Party Licenses

This project embeds or includes (as submodules) several third-party open-source libraries and/or code derived from them. All such libraries' licenses are included in the PACKAGE-LICENSES folder of this project or of the nvpro_core dependency.

Support Contact

Feel free to file issues directly on the GitHub page or reach out to NVIDIA atdisplacedmicromesh-sdk-support@nvidia.com