Uber MDL Translucency
Issue #1091
resolved
This is related to #1075. Credits go to Midnight for getting into the mdl code. I would never ever be able to do this by myself without his insights. Specifically this adds the color effects and normal inversion to translucency that were ignored before.
So below there's the setup and the equations. The first thing we notice is that the shader chain is different, now translucency comes before diffuse, so we also added a daz diffuse group. This is because what translucency does in iray is really to add options to the diffuse inputs, as fac and color. The rest of the chain is the same. Then we added the color effect group and the normal inversion.
# DAZ TRANSLUCENT
# this is first in the shader chain so fac = 1
fac = 1
color = translucency color
if invert transmission normal == on
normal = inverted normal map or bump map
# DAZ DIFFUSE
# note that translucency adds options to diffuse
if base color effect == scatter & transmit intensity
fac = daz color effect: intensity fac
color = daz color effect: intensity color
else if base color effect == scatter & transmit
fac = daz color effect: transmit fac
color = base color * reflectance tint
else if base color effect == scatter only
fac = 1 - translucency weight
color = base color
# DAZ DIFFUSE COLOR EFFECT
# color effects are used also for other things so the group is generic
# the diffuse color effect is meant for fac < 1
fac = 1 - translucency weight
color = base color * reflectance tint
if fac == 0 or fac == 1
prune this node
Below there's the color effect group. Please note that scatter only means no effect, that is, the group is not used.
# DAZ COLOR EFFECT OUTPUT
# for scatter only the group is not used
if color effect == scatter & transmit intensity
output = intensity fac, intensity color
else if color effect == scatter & transmit
output = transmit fac
Below there's the invert n-map group. This has to be used to invert normal maps. Please note that to invert bump maps we can use a simple color inversion. Please note that the texture has to be inverted before the bump or normal map node.
Then below there's the comparison with iray. Test scene included translucency.duf, it's G8F with a translucent skin. It's also included a blend file with the conversion groups translucency.blend, that can be appended or used for reference.
I didn't include scatter only in the comparison because that is no news and worked fine before.
Comments (41)
-
reporter
-
repo owner
A first implementation is ready. A lot of things have changed so there are surely bugs, but at least I have managed to load files with all material method settings without crashing.
-
reporter
Sorry Thomas for the hard work. I really tried to design this for minimal change, adding an extra group for the color effect and using translucency as first in the chain. I mean all the daz groups are the same and the rest of the chain is the same. Then if there’s a color effect that is added. This way I was hoping for the risk to break anything to be minimal.
Well then if I find any bug I’ll report here. Thank you so much.
-
reporter
daz studio 4.15.0.30, blender 3.2.1, diffeomorphic 1.6.2.1082
Ok I found a couple issues. Test scene included test.duf, it’s G8F with a translucent skin.
1. When there’s a white reflectance the multiply node is pruned, so the base color is lost as well, because the diffuse and effect groups are defaulted to black.
2. When merging materials the torso and face don’t merge, though it’s the same material.
3. I managed a better setup for the daz effect group. The equations don’t change so it’s just an internal edit of the group. Please name the color output to “intensity color“, this is important since this output is not used for transmit but only for intensity. At least so far.
4. The new groups are not available in the “make shader groups“ tool.
-
reporter
- attached test.duf
test scene for the mdl translucency, it’s G8F with a translucent skin
-
repo owner
Everything should be fixed in the last commit, I think.
-
reporter
Commit 8539af8 works great thank you. Will leave this open for a while I want to test better.
-
reporter
Ooops .. the diffuse and effect groups from “make shader groups“ are off. Seems they “copy” the fresnel.
-
reporter
There’s another bug. When the translucency normal is inverted all the groups use the inverted normal. Only translucency has to be inverted. The example below is from G8F.
note. Please note that this requires two bump or normal nodes, one with the inverted map and one without.
-
reporter
Not specific to this discussion. But there’s a little “confusion“ in the dual lobe groups. That is, the group fac is the dual lobe ratio, while it should be the dual lobe weight. So they should be switched. We can rename the fac as ratio, then rename the weight as fac and use it as the group fac.
This affects both “daz dual lobe uber“ and “daz dual lobe pbr“.
Also we may rename “daz dual lobe uber“ to just “daz dual lobe“ since all the groups are supposed to be uber unless otherwise specified, this is also congruent with the other names. Same for “daz metal uber“ that can be renamed “daz metal“.
-
reporter
With the same principle we could use the glossy color as fac for the glossy group. Where the fac is actually not connected. If the glossy color is black (or zero) then there’s no glossy and the group can be pruned.
So we can rename the color as fac and use it as the group fac.
-
repo owner
I fixed the issue with make shader groups and the inverted normals. I hope, I only tested inverting the bump map, didn’t find anything with an inverted normal map.
-
reporter
Commit b3b441f works fine for the shader groups and the inverted normals, I tested both bump and normal maps.
Let me know what you think of the glossy facs.
-
You don’t need a “Daz Metal” group at all. Metal is just a type of top coat.
Once the diffuse and SSS are implemented, we can move to glossiness settings. I just wanted to stay focused.
-
repo owner
The dual lobe issues should be fixed now
-
reporter
Commit ee86a4e works fine.
As for the glossy fac it is still not connected. Not a big deal but the user may expect he can drive the glossy with the fac until he opens the group and sees it’s not connected. Would make more sense to either use color for the fac as suggested above or remove the fac input.
Let me know.
edit. p.s. Or of course another option is to add a extra mix to drive the unconnected fac. With a extra mix we can change the equations and use the layered weight as fac and the color as color. While the actual equations multiply the color by the weight that’s the same final effect.
# daz glossy fac = glossy layered weight color = glossy color
-
repo owner
I added the extra mix node. It makes sense that glossy should be handled as the other groups, where the Fac input is the relevant weight. Cleaned up the code a bit, too.
-
reporter
Commit a84ce55 works fine.
-
Hey guys, is there a way to update the material nodes of an already imported char by a button, or do I have to import the figure again and copy the material under advanced options? Tia and have a nice weekend!
-
reporter
update. HOPE FINAL SOLUTION
Ok this one was the death of me I really need a resuscitation spell now, but finally I beleieve I got the color effect right.
Basically the setup by Midnight in
#1075didn't work for dark colors because of the way blender converts colors to scalars that's by luminance. While it seems mdl goes by average. So I used a vector scale to convert the color by average and now both dark and bright colors work fine.
Below there's the new setup and equations. Please note that now the color output is always plugged, then the transmit or intensity fac is plugged depending on the effect.
# DAZ COLOR EFFECT OUTPUT # for scatter only the group is not used if color effect == scatter & transmit intensity output = intensity fac, color else if color effect == scatter & transmit output = transmit fac, color
Below a comparison. There are just some relevant cases to show the issue, everything else works fine in my tests. Please note that the setup by Midnight works fine for bright colors, as the Evangeliya skin.
-
reporter
@ByteC You can use material plalettes. Unfortunately it’s not documeneted yet.
https://bitbucket.org/Diffeomorphic/import_daz/issues/947/make-material-set
-
reporter
- edited description
** corrected the diffuse color effect comments
-
reporter
update.
As for commit bb1f4c1, I see the diffuse group is pruned for overlay = 1. This is good if there’s not a overlay effect (effect = scatter only), but if there’s a overlay effect then the diffuse group must not be pruned bacause the effect needs its input.
I also corrected the diffuse color effect comments in the first post. That is, pruning for fac = 1 is only needed for the diffuse color effect, since for fac = 1 there’s no translucency. This doesn’t apply to other color effects.
Test scene included overlay-effect.duf. Below it’s how it should be converted with the new color effect setup above.
-
reporter
- attached overlay-effect.duf
test scene for the overlay color effect
-
repo owner
Fixed in last commit.
-
reporter
As for commit 0d5d92f, if the effect is driven by a texture then the texture is not connected. Test scene effect-tx.duf.
-
reporter
- attached effect-tx.duf
test scene for effect driven by texture
-
reporter
- attached bw.jpg
and texture
-
reporter
Always commit 0d5d92f, if translucency weight == 1 then fac = 0 and there’s no diffuse nor effect, so both diffuse and the effect should be pruned. No harm if it isn’t though, I mean the rendering works fine anyway.
-
reporter
Always commit 0d5d92f, it happens something strange if the overlay is driven by a texture. The effect is not correctly connected as in the picture below.
-
repo owner
I think I covered everything in the last commit, but the code is starting to be quite complex so maybe something is stilled missed.
But what about the principled method? Currently diffuse overlay nodes are generated with a color effect, and the base color effect affects the principled base color (but the Fac outputs are not connected).
-
reporter
Conceptually the color effect is not complex to handle.
- if a group gets a color effect then the effect takes the group color and fac inputs and plugs back to the group the effect color and fac outputs
- if fac = 0 then there’s no group thus no effect and both can be pruned
- if fac = 1 the effect will take it and change it so we can’t prune the preceding group as we do normally for no effect
As for the principled method, the mdl enhancements proposed by Midnight will take the color effect, the schlick fresnel and the volume. None of these is supported by the principled shader itself, but we can use the enhanced features on the extra bsdf nodes, if there’s any, which will be the same as the bsdf method.
Specifically, the daz overlay is not supported by the principled shader and the actual implementation is a poor approximation. That is, the overlay is chained to the principled shader and will replace it for fac = 1, that’s obviously wrong because this “discards“ any base reflection. But works fine with dual lobe and is not bad for makeups.
The single principled method does not support overlays as expected.
So, as for commit 2d56fea the overlay effect seems well managed, within the principled limits and approximations. The group is added and the effect outputs are correctly plugged. Will let you know if I find any bug.
-
repo owner
Fixed one case that was not handled correctly before: when there is a translucency texture and the translucency weight is less than one.
-
reporter
Commit 83bb9be doesn’t work fine.
For glossy fac = 1 the previous groups are pruned. That makes no sense since glossy is a reflection layer and mustn’t prune anything.
-
reporter
note. important. NO THIS DOESN’T WORK. Nope, this will have the same issue of mixing translucency after principled. That is, a 90% translucent will be 10% diffuse but also 10% specular. So it is true that translucency will match iray, but we lose the specularity. Also this will not work with single principled which can’t use a translucent shader. So we may just leave everything as it is now, since there’s not a neat advantage.
update. TRANSLUCENCY FOR THE PRINCIPLED OPTION
Until now for the principled option we approximated translucency with sss. Because the principled shader doesn't support translucency, and also because it was not possible to mix a translucent shader after the principled shader since this way we lose the specularity.
It may seem obvious, but bringing the translucent shader first in the chain means that now we can use translucency with the principled option. The advantage is that, if a material is translucent, the principled option will convert exactly, instead of using a sss approximation.
Please note that skins (translucency + volume) will continue to use the sss approximation. It is only translucency alone that will be converted with a translucent shader. We can use the skin test in
#1082to check for skins.Below an example with G8F where I turned thin walled on for the skin material, so there's no volume and it is pure translucent. Please note that we can also use the color effect to get an exact match with iray. Translucency comes first in the chain, then it is mixed with principled using the color effect. The equations are the same nothing changes.
# translucency for the principled option principled fac = 1 - translucency weight principled color = base color * reflectance tint translucent color = translucency color
Then below a comparison wth iray. Test scene included translucent-g8f.duf.
-
reporter
- attached translucent-g8f.duf
test scene for principled translucency
-
repo owner
Implemented in last commit. I think the skin becomes a bit too dark, though.
-
reporter
As for commit c6ff828 there’s a couple bugs.
- For a translucent material (no volume) we use translucency so sss must be zero. Actually sss is not zeroed. Then with zero sss the sss color is not used as well.
- For a skin material (translucency + volume) we use sss and there’s no translucency, thus the base color effect is not used. Actually the base color effect is used but doesn’t connect since there’s no mix to connect to. The base color effect should not be used in this case.
-
repo owner
The bugs should be fixed now, I hope. But this is confusing. I was really confused by a test file you uploaded recently, which has translucency but is thin walled, so it doesn’t count as a volumetric skin material.
-
reporter
Commit 71b0af9 works fine.
As for translucent-g8f.duf I turned off the volume on purpose to test translucency. I could have done it with a cube but complex files are better to reveal bugs. That is, skin = volume + translucency, then there’s translucency alone and volume alone that are valid materials as well to be converted differently than skin. As we tested in
#1082.About this in
#1082I added a pbrskin-test.duf for the pbrskin shader that’s not actually supported for the sss option. So it is to be checked with the various material options and fixed as well. Let me know if something is not clear. -
reporter
- changed status to resolved
- Log in to comment
Below there’s an example of using the color effect for another iray channel. In this case the diffuse overlay color that’s used for G8 makeups. The concept is that we take the channel inputs and redirect them to the effect node, then using its outputs depending on the effect we need. This is the same for any iray channel using color effects so the color effect group is general purpose.
Below an example comparison with a modified makeup for Victoria 8.