mixing HD normal maps
Issue #406
resolved
Just tested commit ad5b4cc for the normal maps option, and I see it doesn’t mix multiple normal maps. Below is one way to do this. I added the afraid and angry face expressions with driven normal maps.
I don’t know if Xin or engetudouiti may have a better solution.
Comments (23)
-
-
repo owner
The last commit implements this for both VDM and normals. It seems to work for VDMs, which makes sense since the displacement is just a vector in ordinary space. It doesn’t work for normals, at least not when I tried, but I think that was expected, since the output from the normal map node is a vector in tangent space, I think.
In another place the plugin mixes normal textures with an overlay node, but it is hard to see how to do this if the strength of the morph is not one.
-
repo owner
engetudouiti, thanks for the link. You have probably given it before, and I got the overlay trick from there.
-
Do not mind, I just add other people test experience things, ( I apreciate even if you order me to google something for add on ^^)
-
reporter
Thomas, as for commit f2c9daa I see the setup above is not implemented for normal maps. There’s a overlay setup instead without driven maps. Also I'm not sure to understand why you say that vector addition doesn’t work. Here it seems I get the same results as daz studio.
Anyway if you prefer to use overlays as suggested by engetudouiti, below there’s a possible setup with driven maps. It’s a chain of overlays that ends with the zero color for normal maps that's (0.5,0.5,1.0). It is also more elegant since we don’t need the addition chain.
Then below a comparison. There's the angry + shock combined expression for G8F. I changed the skin to a shiny black to better show the displacement. First iray, then cycles with overlays, then cycles with vector addition. They seem the same to me, apart that vector addition uses strength = 2 to get the same result as iray, don’t know why, may be Xin could explain this.
iray
cycles with overlays
cycles with vector addition with strength = 2
-
reporter
- attached g8f.duf
test scene included g8f.duf
-
I would never add normal maps directly, they are not displacements, but directions. Adding normalized directions is not right and you will get weird distortions in many cases (especially when you start to combine several of them).
In general, adding a lot of normals will tend to wash out the details (as Blender re-normalizes the resulting vector), that’s why you have to increasingly make the Strength bigger to compensate. And it will only get worse with more normals.
For example, if you add a detail normal map to a neutral normal map like (0, 0, 1) (a normal map without any data), you end up with a distorted result that washes out the details and pushes the direction towards the normal direction: (0, 0, 1):
(0.3, 0.4, 0.3) + (0, 0, 1) = (0.3, 0.4, 1.3) == (0.15, 0.2, 0.65) [after re-normalization]
Addition should only be done for vector displacements or grayscale displacements, since these are not directions but displacements.
See this article to understand why combining normals is not trivial (you don’t need to understand all of it to get the point): https://blog.selfshadow.com/publications/blending-in-detail/
The method explained by that article is the best one, both in quality and performance. I would implement a custom node group for it (overlay blending somewhat works too, but it’s more expensive and is less accurate). Although in Blender, native overlay blending vs a custom group could end up being about the same in terms of performance.
The engetudouiti link has an implementation of the method. I also attached a node group in the .blend I added in
#311 -
reporter
Xin, I fear you overestimate my math skills, I’m more a “try till it works” man, but thanks for the link I just trust you. So do you think the overlay chain above with driven facs will work fine enough ?
p.s. Anyway the neutral value in blender seems to be (0.5,0.5,1.0) and not (0,0,1). That is, if I try (0,0,1) for the overlay chain end, then I get weird results.
-
Overlay nodes can work too. They are used often for normals even though they are not accurate. Your set-up seems fine.
The addition is what shouldn’t be done for the normals. Addition should be done for displacements, but not for normals.
And yes, the neutral map is different on the texture, since normal maps are unpacked like this internally:
normal component = (2* [pixel value] )-1. Basically to go from the (0, 1) range from pixel values on the texture to the (-1, 1) range once unpacked, to denote the components of a direction vector in 3d space. You don’t usually need to care about this, since Blender does it internally; and when normal directions are recorded on textures, this is taken into account too. That’s why (0.5, 0.5, 1.0) works as neutral. -
repo owner
Xin, I was busy implementing normal map mixing, so didn’t see your comments until now. Adding the normals did not work, even if the strength of all normal map nodes was set to zero, cf picture. Then I added a normalize node, which fixed that problem but indeed washed out detail. To compensate the strength is multiplied with the number of maps. This may not be accurate, but perhaps for small deviations.
I will look at the article in your link tomorrow. One thing that I’m worried about is how to drive the map strength. I didn’t use the overlay trick because I didn’t find anything to drive.
-
repo owner
After some quick calculations I think that my way of adding normals is not terrible. A normal is of the form n = z + d, where z is a unit vector in the normal direction and d is a deviation. If we just add two normals, we get
n1 + n2 = 2z + d1 + d2 == z + (d1 + d2)/2,
which washes out details. But if we increase the strength by a factor 2, we instead get
n1 + n2 = 2z + 2d1 + 2d2 == z + d1 + d2,
which is what we want. There might be errors hidden in the normalization, but if the deviations are small so are the errors.
-
Basically you start the chain with a normal map that represents no deviation from the normal (in tangent space). That is the typical blue color of normal maps (it’s the default color you get when you add a Normal Map node in Blender, (0.5, 0.5, 1.0)). If all factors are then 0, then that neutral normal color prevails, which would be the same as if you didn’t plug any normal maps.
So basically what Alessandro did above (with overlay mixing).
The node group here https://blendswap.com/blend/13735 implements the method of the article I linked above. See also: https://youtu.be/34BYCkQhHhg?t=865 .
-
btw I do not push any way. because I do not know which way is best for add on, and which way may represent shape key mixing. but basically I think they may show almost same effect. only difference is how mixing with strength I suppose. (eg A normal 0.3 and B normal 0.7 etc)
I think basically there is 2 way,, mixing normal map (R,G,B component) or mixing generate out put normal vector. about both case, we need to get the difference from neutral,
Then to get neutral mesh normal, I may use geometry node normal. because it is what, when we do not add new normal map, the default mesh normal is used.
When I first reply I did not check seriously (I am VDM main user for dhdm ^^; now) but Alessandro first way is not bad. I may use almost same procedure, if I prefer to use out-put vector. (basically I like it more than manipulate texture, so we can keep each normal map node setting as same as before.
The problem is already pointed out, the normal map node out-put vector can not simply add. because it is shading normal vector.
So If I follow first Alessandro way, (use out-put normal vector) I do like this
Which value we may use to control vector strength for shape key value, may change with middle value. Actually I may keep strength value of the normal map node, then may tweak delta normal multiple wth float value, (as vector length) But about any case, so Normal map only controll shading normal, it may not work well when we mixing many map (or many vector) as same as JCM shape key deformation well I afraid. But if use for one dhdm + default normal map for character, may work well I suppose. (do not know, if we mix many hd shape key exression, actually the normal mixing represent daz hd morph mixing)
-
- attached testnormal.7z
pack 2 normal map, I divide 2 surface group, then if you controll each mat you can clear see how border work with 2nd nomral or not
-
And I think if we use normal map strength value to adjust shape key value. we can not represent inverse shape key. (minus value ignored)
Then if I use vector math (scale option), can work well for out-put vector strength. like this
I have not noticed the vector math scale option , but it is option when we multiple float with vector ^^;And if the material already have base normal/bump map, we simply use the normal map as , as base normal. (fist add input)
so base normal (with mat default normal map setting, plug in generate) + (hd normal A - geometry normal) * vector scale (strength) + (hd normal B -geometry normal) * scale (strength)) + c….
may represent each HD shpe key and value(strength), I suppose.
Actually vector math scale node can input minus value correctly, so it work as shape key minus value. (aproximately only) I feel.
-
repo owner
Xin, the way I described above is approximate, but so are the methods in the paper you linked. The output from a single normal map is normalized, viz (z+d)^2 = 1 so d^2 + 2d*z = 0. The sum of two normals is not: (z+d1+d2)^2 = 1 + 2d1*d2, and the last term introduces an error. This generalizes to addition of several normal maps. However,
- The error is small if all deviations are small, i.e. the normals are not that far from perpendicular to the surface.
- The error is exactly zero if all deviations but one is zero, i.e. if only a single shapekey is active.
- It is straightforward to drive the strength of the normal map nodes with shapekey values.
-
repo owner
engetudouiti, do we really need negative shapekey values? Some sliders can certainly be negative, but once we have come to the shapekey the value should be positive, no?
-
repo owner
Now I have contemplated your last picture for a while. It looks very good so far. Will implement it an see what happens.
-
reporter
Thomas, it looks like you didn’t read my post. The overlay chain I posted above works fine for normal maps as Xin also commented, and it’s more elegant than vector addition because you don’t need the addition chain, nor to do math in between. You can drive the fac channels as shown above. Please look at my post with the black faces.
Or if there’s any reason why you don’t like overlays please let us know.
As for commit 6834a01, below it is what I get when using the new “add normal maps” tool in the advanced setup tab. It generates overlays but in the wrong way without the termination chain for the fac channels. Again please look at my post above. Also the normal node is connected both to the bump and to the principled nodes that’s wrong. The normal map has to be connected to the bump only. And the bump has to be connected to the principled normal input.
Then, if possible, it would be nice to add the driven values too when we add the maps. Since the maps and the driven values get the same name may be this is possible to automate, so the user doesn’t need to connect driven values anymore.
steps:
- generate normal maps with the addon by Xin
- import the test scene g8f.duf with the principled option
- “add normal maps” from the advanced setup tab
update. I don’t get anything different with commit f2cd558, it is the same as above. The addon is 1.6.0.0029.
-
repo owner
Alessandro, I read it but it didn’t register. New commit adds textures instead of normals. The overlay node is put in a node groups, whose content can eventually be replaced by the better algorithm.
-
reporter
Thomas, as for commit 078e603 I see we don’t get it. Below how it is now and how it should be, with the exact steps to follow so we have a common ground.
steps:
- Using the addon by Xin generate the normal maps for the G8F angry and shock face expressions.
- Import the attached test scene g8f.duf with the principled option.
- From the morphs tab use “import expressions”.
- From advanced setup use “add normal maps” to import the angry and shock maps generated in step 1. For the face material.
- From advanced setup use “add driven value nodes” for the angry and shock expressions.
Below how it is now the face material. This is bad because we don’t get the fac channels for the driven values.
Then below how it should be, this is good. It would be also nice if the plugin could connect the driven values, may be this is possible since they get the same names as the maps. So the user doesn’t have to. The terminal color in the overlay chain is (0.5,0.5,1.0).
-
repo owner
This is how things look in the latest versions. Seems to be the same as yours, I think.
-
reporter
- changed status to resolved
Thomas you got it commit 765a228 works great, marking as resolved.
- Log in to comment
I can not confirm,(I do not know clear) the vector add option can be used for mixing normal map, but if it show reasonable mixing effect, (as same as daz), no problem?
about displacement, if we use out-put vector (used as displacement input of material) it actually show delta vector, how move verts from default position. so I may simply add (sum) those vector. for me it seems almost same how multiple shape key added with each value then deform mesh.
But I may test it actually it can be mix 2 dhdm well or not.
About normal map mixing, there are many user aploach.
https://blender.stackexchange.com/questions/38298/how-to-combine-two-normal-maps
Then I do not know which way may best for add on. (as my purpose, I download the mix normal node from blend swap. just for add new normal map (like tatoo etc), then do not think seriously use it for shader shape key.