Nanites missing Piece

In my previous blog post, I explained the capabilities and limitations of Nanite, including the lack of traditional vertex color workflows. This limitation is constraint to how Nanite works. Mesh clusters and internal usage of triangle clusters on an internal LOD basis make vertex colors unreadable without baking them into the prop. Additionally, these colors flicker due to cluster changes on a frame-by-frame basis.

Today, I'll continue with a solution that not only is a workaround but also enhances the creative process: Material Pixel Paint Decals.

Recap Vertex Painting

Vertex painting assigns color data to every point (vertex), offering three gradient maps (RGB) on the mesh data at no extra cost. Vertex Painting is widely used for various purposes, including stylized low-poly effects, baked shadowmaps, VFX, and material blending.

While vertex painting for VFX isn't problematic, as it's not intended for rendering in Nanite, the technique is crucial for blending material layers in environments to break up tiling and add weathering and damage effects.

PROS

Relatively easy to use
Real-time feedback
Quickly adjustable color data
Can procedurally baked in Houdini

CONS

Changes to geometry destroy vertex paint data
Relied on underlying geometry tessellation
LODs are a problem
Nanite limits baking per Module and can cause flickering

Current Workarounds and Limitations

There are two common methods for achieving gradient maps for material effects in Nanite

Vertex Color baking per Module

Nanite utilizes vertex colors (procedural or painted) by baking them into the mesh for optimization. However, this requires high-resolution meshes and suffers from:

Flickering/Reduced Color: Nanite's per-frame optimization can cause issues, especially on flat surfaces due to distance-based polygon clustering.
Limited Baking: Baking is per module and requires re-baking for vertex color changes and does not take environmental setups into account.

Texture Baking per Module - The Matrix Example

In the Matrix demo, Epic addressed detail variations in Nanite by baking texture maps for each material module, enabling distinct variations per instance. This technique is practical for hero assets that require a lot of detail. However, it's important to note the process can be heavy on resources, workload and might affect performance.
Despite that, it remains a solid choice for procedural variation on a prop basis, to balance detail and performance.

A new Workflow - Material Blend Decals

Leveraging DBuffer Decals

To overcome the limitations of Nanite, a new approach is needed. This approach should empower artists to paint environmental details and offer independent control over material blend layers, acting independently across multiple objects.

To address this, I want to Introduce you of the Material Blend Decal Concept.

This solution leverages an underutilized channel within DBuffer decals: the specular channel. In modern PBR workflows, materials derive their reflectivity from metal and roughness values, rendering the specular channel somewhat obsolete or at constant variables from 0-0.5 .

As in most cases there is never a usage for a Decal to have specular Information, we can repurpose the specular channel of the Decal Buffer to drive a gradient map for environmental material blending, like dirt or grime.

Download the Asset Pack on Unreal Marketplace

PROS

Pixel perfect Material Blend on across multiple Surfaces
Independent Data Layer to Manage details separate from scene geometry.
Can be procedurally placed and authored
Can be placed in prefabs & instanced
No LOD flickering
Mesh decals can provide advanced shapes
Can be animated

CONS

Decals will not add specular on surfaces - which is fine for 99%
Just one gradient map - two are possible when decals don't need metallic
Decal Textures Resolution should be capped to 265px-512px to be game performant

Material Blend Decal Overview

Material Blend Decal Application

Keeps standard functionality with other PBR Decals

Comparison

Vertex Paint Workflow

Material Pixel Paint Decal

Where we are in the Deferred Pipeline

PBR Pipeline and Specular

With PBR workflows, the specular channel is often not used at all and set to the constant of 0.5 for decals. For Metallic Values the specular channel is not used at all.

By hijacking this channel for our decals, we effectively utilize this "free" data stream to carry our gradient maps for environmental textures. This reassignment, however, means that any materials using this custom decal response can no longer be affected by specular decals.

General DBuffer Decals

DBuffer decals are evaluated for lighting and shading across their specific data channels in the deferred rendering pipeline. These decals typically support only Translucent and Alpha Composite blend modes, and can be tailored per material via three distinct DBuffer textures and channels.

DBufferA:
Color RGB - A: Stores base color information (RGB) and alpha channel (A).
DBufferB:
Normal RGB - A: World normal information (RGB) masked by opacity alpha (A).
DBufferC:
Roughness (R) - Metallic (G) - Specular (B): masked by opacity alpha (A)

For Mesh Materials the DBuffer decal response can be set individually on these Channels. These nodes read texture data from the DBuffer directly into the material graph, providing customizable flexibility.

Pipeline Setup

DBuffer Decals can be used for a wide variety of blending and material manipulation. I aimed to maintain all the usual decal behaviors, including dynamic gameplay applications like blood splatter and other common decal uses. To ensure compatibility with all pipelines, the Asset Pack only utilizes the specular channel, which is typically unused 99% of the time.

Depending on the trade-offs you are willing to make with the provided channels for other decals, you can further customize this approach. For example, if metallic decals are not required on certain surfaces, that decal channel can also be repurposed to create a second blend layer.

Important Steps for custom Setups:

Disable DBuffer response in the Surface Materials you want to use Decal blending
Implement custom DBuffer response Material Function for all Materials you want to use it.
Copy the Blend part and use it for your custom Blending
ALL packed Displacement and Gradient Maps needs to be set to Mask compression in the Texture settings
The Material Blend Decal needs to have a Specular Value set at 1 to enable proper blending.
Set priority order to -1 as the Material Blend Decal needs to operate below all other decals to work as expected.
For all other decals, set the specular value to zero and layer above blend decals, unless a specific effect is desired. E.g. blending from a clean surface through dirt puddle, the specular should also be set to 1

Edge Cases:

The way Epic designed the Decal Render Pipeline means that writing to the specular channel will also write to roughness and metallic channel and vice versa. This design is not ideal as it leads to complications, causing issues with how all sort decals are used together.

For our Blend-Decal there are some edge cases where you see a harsh line on roughness blending. This can be solved by adjusting the roughness value of the Blend-Decal close to the surface roughness you are applying the blend on.

A Simple Blend-Decal

For this setup, we want to write data specifically to the DBufferC channel, focusing on the Specular channel. We set the Specular channel to 1 to classify the decal as a Pixel Paint Decal. Additionally, we want to integrate a decal shape for pixel-perfect material transitions.

Note:

This showcases a basic material setup in Unreal Engine, while the Asset Pack offers a more versatile setup with enhanced gradient, opacity, and surface alignment functionalities.

The Surface Master Blend Material

Disable Decal response

In the Master Material itself first we have to disable the response to any decal on the overall settings. After that we manually can reactivate decal response and setup each channe manually.

For that I have made a custom Decal Response Material Function which can be added right before the final output to the Material Attributes.

Custom Decal response

DBuffer A - Albedo:

RGB channels of the decal's albedo are linearly interpolated over the decal’s opacity alpha channel to blend with the material’s base color.

DBuffer B - World Normal:

For world normals, RGB maps are similarly interpolated using the alpha channel to set decal normals onto the material surface, influencing lighting and shading behavior.

DBufferC - Roughness (R) / Metallic (G) / Specular (B):

Here we just want the Roughness and Metallic channel. Again a lerp is used and we plug this stream in Alpha B. This gives us back all decal PBR functionality we need for all kind of materials.

However, enabling these channels in the decal also causes the Pixel Paint Decal unintentional writing their Roughness channel on the surface which should be controlled by the material itself.

To manage this, the Specular (B) channel’s value is used with an inverted mask to yield a binary response that flags the presence of a Pixel Paint Decal.

By dividing it with an inverted mask, we generate responses of 1 wherever the Specular channel is active due to the Pixel Paint Decal. This condition is then used in an if-statement to decide if the Material’s Roughness/Metallic should be overridden or if other decal types, like blood, should be considered.

As the roughness and metallic values are as well premultiplied by the Alpha we have to make the same inverted alpha and division operation to apply the correct values to each output.

Material Texture Blend Setup

In the Master Material setup, we use now the Specular channel as the material blend channel. Activating one parameter within DBufferC will cause the others to be automatically evaluated using their default values.

Meaning other decals with 0 or 0.5 specular are also writing in the specular channel(even when not intended) and we have to calculate out these values. This limits the available greyscale gradient range, but adjusting these values can still produce an effective gradient for blending

We now can use this mask in our Height Lerp or any other Material layer Blend

The Asset Pack

Asset Pack on Unreal Marketplace

Materials:

1 - Surface Master Material with Height Blend Setup - ready to work with Megascans Texture - and several instances for testing

1 - Material Function for custom Decal response and several Instances to test with

1 - Material Blend Decal with advanced opacity and gradient controls.

1- Surface alignment Material function

1 - Debug Decal Material to visualize the maps

1 - Puddle Decal to test it within a standard roughness decal scenario

Decal Texture-sets (gradient Maps):

57 unique Alpha Maps across 9 Categories - 1024px

Each Category has 5 or more Texture maps in 1024 resolution - for performance focused scene reduce the decals to 256 or 512 and pack them in RGB Channels.

Environment Texture-Sets (Albedo / AO, Roughness, Displacement / Normal):

6 Surface Texture - 2048px:

Performance

A big factor in using DBuffer decals as part of the raster render path is that they require depth prepass to be active in the scene. In practice, for higher end platforms this isn't much of an issue because systems like Virtual Textures, Nanite, and Lumen all use the depth prepass. If any of these are being used in your project, there's no penalty to using DBuffer decals.

When it comes to optimizing performance of DBuffer decals, it's best to only enable the DBuffer decal response for materials which actually need to receive decals, which includes enabling the channels in the receiving material that are actually being used (see Decal Response). Each material which utilizes the DBuffer has additional code to do so which in turn adds to the complexity of the material.

There can also be some cost associated with reprojecting the previous frame's normals, but only for the sampling and reprojection of the pixels touched by any decals.

Further possibilities

If your Decals are not required to overlay Metallic channels we could also repurpose that channel to get a second blend layer.
The biggest limitation is that the DBuffer Decals are writing in all Channels of the Roughness, Metallic and Specular even though just one of them is needed.
If you have the knowledge of graphic programming and run a seperate branch of Unreal Engine and are able to change a vector to a single float approach it would open a lot more possibilities.
Another Addition would be to split the gradient range in two from 0-0.5 and 0.5-1 but this would require a very custom alpha map setup.

Conclusion

Key Takeaways:

Nanite's bake per mesh approach and reliance on triangle clusters within internal LOD management makes vertex colors impractical.
Material Blend Decals leverage the specular channel in DBuffer decals to achieve pixel-perfect detail for environmental material layers.
This approach offers independent control over material blends, overcoming limitations of baking vertex colors directly into Nanite meshes.
Material Blend Decal advantages:
- Pixel-perfect detail even with lower-resolution textures.
- Independent data layer for managing details separate from scene geometry.
- Procedural placement and authoring capabilities.
- No LOD flickering issues.

Benefits for Developers:

Enhanced artistic expression and control over mutliple modules details within Nanite workflows.
Improved efficiency by separating material details from scene geometry.
Potential for future advancements with support for more channels in DBuffer decals.

Add comment

Comments

There are no comments yet.