This is the norm - 3: normal map types

Part 1: what are normal maps and how do they work

Part 2: how are normal maps baked

Like many other things in our industry, normal maps have evolved over the years, and today there are several types that can look different. In the article I will list those that I remember, but maybe there are others.

Map of the tangent space normals (Tangent space normal map): the most common type today, normal maps; that is what we talked about in previous articles. It modifies the direction of the normals of the model based on the direction of the normals of its vertices (that is, we need to control the normals of the vertices of the lowpoly model).

Normal map of the tangent space Mikk (Mikk tangent space normal map). Not all 3D editors calculate the average of vertex normals the same. This leads to the fact that the appearance of normal maps is different in different engines, so we need to bake the normal map using the same method that the rendering program uses (this is called "use a synchronized workflow ( synched workflow )")

Mikk proposed a way to calculate vertex normals, which was supposed to be universal so that all programs calculate them the same way. From the point of view of the workflow, this means that you can use the low poly model with all its averaged normals (with one smoothing group or with smoothing all the faces), bake the normal map in the tangent space of Mikk, and it will look exactly just like a high poly model, without the need to eliminate smoothing errors or separating hard edges in UV. In the future I will write a tutorial on how to do this.

Remember that this is still a map of normals of the tangent space, but the normals of the model are calculated in a universal way and the models can be used in different programs.

Two -channel tangent space normal map: it turns out that using the information stored in two of the three channels of the normal map, the computer can calculate the third, reducing the memory footprint at the cost of increasing the number of calculations. Since memory is usually in greater shortage, this optimization is often used and some engines execute it automatically (for example, Unreal Engine, when we set the “normal map” parameter to compress texture normal). Having freed one channel of the normal map, we can reduce the size of the texture or use this channel for metalness / roughness / opacity ...

Normally, the blue channel of the normal map is eliminated, so such textures look yellow. Since this optimization is sometimes performed automatically by some engines, you can notice such textures in your project.

Map of the world space normals (World space normal map): this card is normal, instead of modifying the direction of the vertex normals, completely ignores them and changes the way light reflection lowpoly-model in world space (world space) (when baked it considers that vertex normals parallel to the axes of the world).

We can say that the normal map of tangent space tells the model "you must reflect the light to the right," and the normal map of world space - "you must reflect the light to the east."

Such normal maps are more colorful and have more noticeable gradients; they were used because in this case you don’t need to think about the normals of the lowpoly vertices, but they have a drawback - you can’t move the model because it will look strange (we set the face so that it always reflects light to the east. If we rotate it, then the edge will continue to reflect light to the east.).

Today, normals of world space are rarely used in games, but they can still be used to create beautiful textures, for example, the blue channel shows how the model should reflect the light falling on top of the model, so you can use it to add color lighting to the texture.

It is also worth remembering that the world coordinates in different applications are implemented differently: in Unreal, 3D Studio Max, Blender, the Z axis is directed upward, and in Maya, Modo and Cinema4D, the Y axis is directed. This means that when transferring world normals between applications spaces may deteriorate.

Object Space Normal Map(Object space normal map): This is an improved version of the previous type of map, and it is very similar to it. The idea is that when you move a model in the world, its normal map of world space should be reoriented relative to the object.

This can be described as "this face should reflect light to the right of the model." If you rotate the model in the world, then the normal map should change in accordance with these changes. However, this does not work with deformable meshes, because in such maps only the movement of the object is taken into account. It is for this reason that the normal maps of tangent space are most common today.

Inclined Normal Maps(Bent normal maps): in essence, they combine AO ​​information and normal maps, tilting the directions of the normals so that the light tends to reflect to those parts of the model that the light hits.

Such cards are used to enhance Ambient Oclussion and to avoid an effect called “light leaking,” in which a model can reflect light in parts that it cannot reach. Personally, I have never used them, but I would have explored their capabilities if I had encountered a noticeable "light leak." More information can be found here , here and here .

16-bit normal maps(16 bit normal maps): sometimes when there is a very smooth gradient on the normal map, we can notice the appearance of stripes. These stripes are due to lack of colors to represent a smooth gradient, usually caused by texture compression.

But in the case of a large and smooth surface, such problems can arise even with an uncompressed texture. In this situation, you can use 16-bit normal maps, usually in .tga file format, which have more colors and are larger than regular 8-bit normal maps.

You can learn more about 16-bit normal maps at the tutorial god himself - Earthquake .

It should also be noted that other techniques exist to reduce the consequences of this problem, for example, completely eliminating normal maps (only geometry is used to represent this smooth surface), transforming lowpoly so that it looks more like highpoly so that gradients are less noticeable, or the use of dithering.

So which type should we use?

In 90% of cases, the best solution is the normal maps of the tangent space Mikk. Unlike options using normal maps of the space of the world or an object, the model will be able to deform, and the direction of the normals will remain correct.

You should bake the normal map in the same tangent space as in the rendering program. The most common tangent space is Mikk, so use it whenever possible.

If pixelation appears on your normal map, consider using 16 normal maps or one of the solutions mentioned above.

In fact, these are all types of normal maps that I could recall. If you know of any other types, then let me know about them and I will add them to this tutorial!

Thank you for reading, I hope the article was useful to you. Thanks to Shnya for comments and help.

Source: https://habr.com/ru/post/undefined/