😦 👩🏽‍⚖️ 👽 This is the norm - 4: solving problems with normal maps 👨🏿‍🌾 📜 🧓🏼

Part 1: what are normal maps and how do they work

Part 2: how are normal maps baked

Part 3: types of normal maps

This is the fourth part of the normal map tutorial, but it is independent of the rest and can be read separately.

Here I list the problems of normal maps that I have encountered over the years of work, and some solutions to resolve them.

Problem: on the edges of the model there are "black lines" or "inserts"

This happens when the model has hard edges, since the vertices of the model have normals that are completely perpendicular to the surface of the polygon, which is why the baking program may skip some details (leaving black lines on the model).

Solution: Normal map baking programs take this problem into account when creating normal maps and try to reduce its influence by calculating a little more information behind the vertex normals, but to save it, they need a gap between the UV-scan polygons.

There is a more detailed explanation here, but in practice the rule is quite simple: when the model has a sharp edge, it is necessary to separate the edges connected to it in UV.

Problem: The normal map looks TOTALLY wrong, especially at some angles

This problem can occur for a variety of reasons. Let's discuss some of them:

You are using the wrong tangent space: the normals of your lowpoly model, which we are trying to modify using the normal map, can be calculated in the baking program differently than in the program used to render the model. If these calculations are different, the normal map can look very strange, especially from some angles.

, . , , .

: Mikk. , . Mikk, handplane .
-: — . , . - — , . - , linear/linear color/raw sRGB Unreal.
: , , (bump map), (displacement map) ....
lowpoly- lowpoly : , / (smoothing groups)/ (hard edge), / , .

lowpoly , /, ( obj ) .

: ?

Solution: you don’t need it. It is not necessary to create normal maps for everything. A sharp cone is a classic example of this, but there are many other cases where normal maps are not a solution.

We use normal maps to change the direction of the normals of lowpoly models. Sometimes the direction of the normals is correct and does not require any changes, and sometimes the lowpoly normals are extremely rotated (as in the case of a sharp spike) and the details from the highpoly model are incorrectly mapped to the surface of the lowpoly model. In such cases, I simply delete the details of the normal map using this color:

This color is 50% red, 50% green and 100% blue, and it does not change the direction of the normals of the normal map of the tangent space, so you can use it to remove parts in places with an unsuccessful projection.

A sharp cone is just an example of one of the cases when normal maps may not solve your problems. It is important to remember here that in some cases the normal map is not the best solution, its capabilities are limited and we cannot expect that they will behave as we need in any situation. Sometimes we spend a lot of time trying to get the normal map to work, when we could just add parts to the diffuse texture or lowpoly, and not try to transfer the part using the normal map.

Problem: the details on my model look inverted.

This is a very common problem and can be seen in many video games, even at the AAA level.

As we saw in this tutorial [ translation on Habré], normal maps are textures that use green, red, and blue channels to change the way light is reflected from the model’s surface when it falls, respectively, from the side, top and front of the model (remember that this is simplified explanation and it is not 100% true).

The problem is that some applications believe that the green channel should show the model as lit from below, while others believe that it should show the model as lit from above. This is sometimes called the “normal map right-handiness”:

OpenGL-applications (right-sided, positive green channel): Blender, Maya, Modo, Toolbag, Unity.
DirectX-applications (left-side, negative green channel): 3DStudio Max, CryEngine, Source Engine, Unreal Engine.
Substance Painter can work with both types and export both types of normal maps.

Solution: invert the green channel of the normal map. Most game engines have an option to texture invert the normal map, or you can manually invert the green texture channel in Photoshop (go to the channels tab, select the green channel and press Ctrl + i).

Problem: some parts appear flat / missing parts

Imagine that when baking normal maps, the baking program emits rays from the surface of the lowpoly model, following the lowpoly normals, until the rays intersect with the highpoly model and rotate. After that, the baking program takes this information and saves it to the normal map.

These emitted rays should not move endlessly, because they can collide with the far part of the highpoly and rotate incorrectly, so the baking program limits the range of emission of rays and sometimes the rays can stop before they can intersect with highpoly. In this case, we lose details and the normal map has areas of a flat color.

Solution: depends on how the baking program allows you to control the baking distance:

lowpoly , «» lowpoly- ( ). , lowpoly highpoly.
, Max, (cage) — «» lowpoly, .
( Substance Painter max frontal/rear distance).

You can also try to make lowpoly and / or highpoly more similar to each other so that the rays can get details across the entire surface of the model. Another option is to bake two normal maps with different cage distances and mix them in different parts of the textures. Some normal map purists may swear at you for this, so take care of your ears.

Problem: Normal map has distorted details

This is a very common problem. This happens when the lowpoly normals do not correctly correspond to the details of the highpoly, so they seem tilted (in fact, if you look at them from the direction of the normals of the vertices , they correspond perfectly). This usually happens when some faces form a very sharp angle.

Solution: I wrote in more detail about this topic in another tutorial [ translation on Habré], but in general the solutions are as follows:

Smooth the sharp corner by adding a bevel.
Convert an edge of an acute angle to a sharp edge (hard edge) / divide the faces into different smoothing groups (smoothing group).
Use custom normals / weighted normals.

Problem: The normal map looks pixelated or contains stripes

Earthquake (AKA normal map god) wrote a very good explanation for this problem.

If lowpoly and highpoly are very similar, then most of the normal map will have one base color of the normal map, and the other colors will be in the places where lowpoly differs from highpoly.

If the situation is the opposite: lowpoly and highpoly are very different, then the normal map will have a much wider variety of colors and gradients will appear:

Such smooth gradients cause problems because many colors are needed to represent them, and the most common texture compression methods are based on reducing the total number of colors.

Solutions:

lowpoly highpoly: , . lowpoly, highpoly.
16- : 8- . , 8 (0 1). 256 .

, .

16- 16 , 65536 . , . , 16- , 8- ( ). , 16- -; 24- . [. .: .]

, , 16 .
: — , , — . , , . .
, : , , . , : , . . , ( , , ), .

:

The obvious solution would be to increase the UV- “island” of this part of the model or use larger textures, but let's look at less obvious solutions:

Bake the normal map with doubled resolution, and then reduce the image size: if you use a 512x512 texture, bake the normal map with a resolution of 1024x1024, and then convert the image to 512x512. Due to this, each pixel of the final texture will receive information from four pixels, creating a kind of antialiasing and reducing pixelation. This is also true for other baked images; In addition, it is worth saving high-resolution texture versions in case you later want to increase details in some areas.

, , , 1024, highpoly, .
UV islands , . 1 UV-, .

trim- .
, . - , . :

Problem: my model is symmetrical, but the normal map is different on different sides.

When applied to a symmetry model, the directions of the normals can change due to the fact that the way of connecting the faces has changed. Sometimes this means that you can get a seam right in the center of the model. To avoid this, make sure that the lowpoly normals in the center are aligned, and adjust the smoothing if necessary.

Another possible reason is triangulation : when models are imported into the game engine, they are always triangulated and sometimes this process can change the lowpoly normals, and artifacts may appear along the diagonals of the lowpoly faces. To avoid this, triangulate the model before baking, bake the normal map, and then apply the symmetry modifier.

In addition, you can study the small Earthquake tutorial , which helped me a little more to understand vertex normals and normal maps.

For more information about normal maps, see the Polycount wiki.

This is the norm - 4: solving problems with normal maps