New commit brings back the incorrect refraction mixing while using Principled for both opaque and refractive surfaces

[Xin]

Issue was solved in #282 by using double principled nodes for mixing, and that would still be the natural choice for this method if you are choosing to use principled for opaque AND principled for refractive surfaces.

Test with V7 and look at the Cornea to see the Issue:

The problem is that it ignores the color map for the Cornea, and that it also uses the refraction map for mixing opaque/refractive incorrectly by plugging it into the Transmission input of a principled node. It should NEVER be plugged into the Transmission input. You need to mix the opaque and refractive surfaces with a mix shader. In this case, both the opaque surface and the refractive surface should be based on a principled node (one principled node for each one, making it two nodes).

---

I will clarify what is starting to be a misconception that might be confusing Thomas too:

• Refractions maps are not the source of any problem if used with Mix Shaders. The problem is when you use the Refraction map in the Transmission input of the Principled node. That’s extremely wrong for reasons detailed in the issue cited above.
• Now all the possibilities for mixing opaque/refractive portions are the following (this is the mixing that a refraction map does):

1. Principled for Opaque, BSDF for Refraction. How do you mix them? with a Mix Shader of course.
2. Principled for Opaque, Principled for Refraction. How do you mix them? STILL with a Mix Shader (yes, 2 principled nodes, that’s not an issue at all).
3. BSDF for Opaque, Principled for Refraction. How do you mix them? STILL with a Mix Shader.
4. BSDF for Opaque, BSDF For Refraction. How do you mix them? STILL with a Mix Shader.

‌

[Thomas Larsson]

For a long time only cycles could handle the bsdf method. Then the meaning was:

• BSDF = Cycles
• Principled = Eevee (and cycles)

With that philosophy it was crucial that both methods were as faithful to Iray as possible. But now eevee handles the bsdf method as well, and the new meaning is:

• BSDF = Faithful
• Principled = Simplified but possibly imperfect.

If you intend to use the loaded materials as they are in Blender, there is no reason to use the principled methods anymore. If you want a material that can be easily tweaked, use the principled method, but then you need to know what you are doing.

Another aspect is what happens if you want to export your character to another app, e.g. a game engine. I don’t know anything about game engines, but I imagine that exporters would handle a single principled node well but will have problems with general node trees.

‌

[Xin]

The current handling of opaque/refraction mixing is unorthodox and is introducing weird additional code that makes the shading look worse (even in Cycles, it’s simply wrong from a PBR standpoint), while making the code less consistent, with principled refraction being treated as a special case.

For portability issues, refraction is already broken since you are using SSR for Refraction, a feature you will rarely find in other software so the result will vary widely, defeating portability. You also end up with missing information if you plug the refraction map into the transmission input (for example, there is no color map at all in the example I posted, even though the map exists), so portability is even more broken.

The proposed consistent handling would make the code cleaner too, and would resemble how shaders are written and combined. I don't see much of a reason not to implement this kind of cleaner logic that also gives PBR results:

refractive_bdsf = None
opaque_bsdf = None

If Principled Opaque:
    opaque_bsdf = principled_opaque_node()
else:
    opaque_bsdf = bsdf_opaque_group()

if Principled Refractive:
    refractive_bdsf = principled_refractive_node()
else:
    refractive_bdsf = bsdf_refractive_group()

mix(opaque_bsdf, refractive_bdsf, refraction_map)

‌

[Alessandro Padovani]

Xin, portability is not intended to be fully supported, it is the other side that needs to be compatible with the principled shader. As Thomas explained, we only care about keeping the principled shader simple to help portability. Personally I know that prorender and gltf are meant to be compatible with the principled shader. Don’t know how much. Sure some fixes will be needed.

As for refraction maps I agree with Xin and Takeshi here. That is, either use dual principled or ignore the refraction map. We can’t plug the map on a single principled node. Then ignoring the map will force users to use the bsdf option for complex refraction, that may be it’s what’s intended in the latest commits.

p.s. @engetudouiti , is you name Takeshi right ? It’s just that engetudouiti is hard to write and keep in mind.

[engetudouiti]

@Alessandro

I do not worry even though my real name is appear hear or there, (I not care about it,, I easy think my link often appear my handle name for each forum,, in blender forum or dev forum or daz forum,, (kitakoredaz in daz forum before), or use tokikake etc,, but basically not hope to use real name.

actually once I set real name as ID with some forum (then when I use real name as ID usually my windows mail box really get huge dumb mails ^^;

Then may not hoped to called by real name in PC world (for hobby things related) you can call me as you like engetu ok kit OK etc but takeshi no ^^;

[Alessandro Padovani]

I also believe we have to decide or understand a direction here, what do we want from the bsdf and principled options ?

A. It is a fact that iray can’t be translated fine to the principled shader, because of limitations with the principled shader. For example about volumes, translucency and colored reflections. These are limitations with the principled shader itself, both with cycles and eevee.

B. It is also of course a fact that iray can’t be translated fine to eevee, either with principled or bsdf, because of eevee limitations.

C. It is a fact that we can convert iray to cycles quite fine if we use bsdf nodes. There are issues there too of course, but it works fine enough to be considered faithful.

It is because of the three points above, that are facts, that the best we can do is below.

‌

1. bsdf: cycles full iray materials, eevee limited iray materials (because of eevee limitations)
2. principled: cycles and eevee limited iray materials (because of principled and eevee limitations)

‌

So, since both eevee and principled are limited as for iray materials, my personal vision was we can simplify as below. Where the user is expected to use bsdf with cycles for full iray materials, and principled with eevee for limited iray materials. Then the user can fix principled materials as needed, since fixing principled is easy anyway.

edit. Then I accept and help whatever Thomas decides of course. This is just my personal vision.

‌

1. bsdf: cycles only, full iray materials
2. principled: cycles and eevee, limited iray materials

‌

p.s. @engetudouiti I’ll avoid Takeshi then, it was just easier also to give credits.

‌

[engetudouiti]

My opinion is,, 2 is OK, but I may try to keep get reasonable effect then may keep request, if it think useful with principled bsdf (as base)

refraction (glass) type shader is usually offered as dffirent,, but still principled bsdf is stable and reliable for most of case (though I know difficulity ehough to auto-convert iray uber (daz custom) so, even though try to make it better, it often cause new issue with daz shader option.

Then about BSDF, I hope we can keep enhancement it as separate group node. If Thomas think it is annoying, of course I follow,, so use BSDF mode only cycles.

But I suppose Thomas may not think it is issue (divide as 2 group for BSDF) .

So my understanding is,

bsdf usually offer more option correctly convert (try it) for cycles and Eevee.

then principled version may sometimes lost some option, and may effect some case, but usually show reliable skin or mat.. (about other prop detail, then if you like customize, or make texture actually principled bsdf use as base,, far easy,, about refraciton things, many paint app not support full.. (they so not color glass,, but use alpha only so I can never paint glass with 3d coat,, (I do not have subsstance painter, then do not know,,) and if you blender fleak,, you may hope to use paint in blender, then principled bsdf as base should offer more tutoriall etc,, I suppose..

If you customize textuer, material often BSDF option already complex enough, I think..without you follow all topic when it have added. or changed.

about principled bsdf, we may need not so many new custom shader I suppose.

Then after all Eevee < cycles, basically,, to make it better we need many scene work. we do not try to include these things to convert iray shader I think.. so there may be many place we can improve it.. (I do not think I can enhance this though)

https://docs.blender.org/manual/en/latest/render/eevee/light_probes/index.html

What I means,, so I do not think BSDF should be offered for cycles only. we just need to remember, maybe BSDF can easy cause issue with Eevee. because of it include many custom nodes etc,, (which Eevee do not support full)

[Xin]

Fixing principled is not really easy when the current broken set up is ignoring entire textures. It’s simply broken right now. Portability, even if it worked, would also be broken, it’s missing a texture.

I don’t really understand this idea that somehow some unproven “portability” to software other than native Blender must be maintained. Have tests been done?

Consider the proRender node:

It has a separate Refraction Color, so the portability is already broken since the Principled node has no separate color for refraction (that’s why the addon is ignoring the Color Map). In fact, no differentiation between refraction color and diffuse color is the reason why we used a second principled node.

It would be far more useful to any user to actually have the Color Map available and plug the map manually to port it to the proRender node properly, instead of having no idea what’s wrong because the addon won’t use a Color Map. You would have to go back to daz to understand what’s wrong.

It seems very sloppy to keep such a broken set up hoping that some unproven automatic portability will work (it won’t, it’s missing a texture). The addon should prioritize working well first. If it doesn’t, of what use would portability be? ignoring textures entirely is just wrong. At that point, just drop support for principled refraction.

Other than that, I like the current Opaque/Refractive separated options more than the old ones, it gives the most flexibility (if they worked properly) and code-wise they are very clean to maintain, as I described above (no weird special cases like the current Transmission input for principled mixing). If you have ever read MDL shaders, you would know it’s quite similar. Encapsulating Opaque and Refraction is the correct choice. No reason not to expose those options to the user too.

[engetudouiti]

Sorry,,, Today I only play with over-ride things, then did not check up-date shader still^^;

I may add something,, after see how each option work,, (but I am a little bit tired to think about node gorups ,, so my head may not work well,,)

[Alessandro Padovani]

Xin, prorender can use the principled bsdf directly, then I don’t know how good it is. I never tested prorender or unreal via gltf myself.

[engetudouiti]

wao about how think option, I perfeclty agree Thomas view.

Then lets remember, eevee offer some only eevee shaders, and set ups (like brobe)etc, we do not test them and use them flexible at all,, then if we try eg

https://docs.blender.org/manual/en/latest/render/shader_nodes/shader/specular_bsdf.html

or https://docs.blender.org/manual/en/latest/render/shader_nodes/converter/shader_to_rgb.html

it should be used for BSDF option, do not think so? so there is no need to use one BSDF set up,, I may call,, principled option and Custom option (which Thomas try to test or add new things etc,, to make shader gogeous, but shader group may perfeclty different, eevee and cycles for mat (because eevee may need different scene setting or shader)

I actually not care even though we merge prinipled shader in BSDF node, if it work well for particular case.. , (I may call it as BSDF option)

on the other hand,, principled option may keep it as base.. (then add something when it need,, with limit)

Then as my default thinking,, I may not seriously render, with eevee , but only need as viewport, (and as for me it need to render some preset icon,, so eye is important, but not expect complex scene reflection etc,.. then had not requested about eevee.. about other material things.. (do not have knoledge to talking about it fully)

‌

[engetudouiti]

I believe OP report should be simple bug,, I do not know why (we had talked how mix base = diffuse main usually with refraction (or transparent) old problem return..

[engetudouiti]

I do not think yours agree to offer 2 X 2 options only for transparent and refraction.. I actually can not imagine how each option work at all ^^;

divide transparent and refraction seems not meaning for me… user simply hope to use work better for each render engine…so it need plug in vendor decide,, which shader you may recommend..

if they understand, thin wall not work well if we mimic by principled bsdf only, they simply hope to add another shader..

eg I simply hope to generate,

base prinicipled bsdf (I hope it as base to get base diffuse and base glossy ), but then divided refraction shader, (which may show ior reflection with coat) which option I need to select?

I think current goruping way, user may not understand,, how you document well, I suppose ^^;

I may agree we need 3 option if it need, but can not understand, refraction (principle or bsdf), transparent (principled or bsdf),

because how merge them may differ with import suface…

if it only need glass shader,, maybe I can understand,, but we already talk about mixing surface, (diffuse part with refractive part divided by refraction weight)

reasonable option may be

base = principled,, refractive(transparent)= principle

base = priniciple refractive(transparent), = custom

all = bsdf custom nodes group (or free vendor recommend set up)

anyway we need to forget to use one principled bsdf to make work all.

we may agree (about this case,, choose 2 or 3 is enouogh),, then if we must need clear coat refleciton.. I offer,, it made (divided() from pure principled bsdf, and it perfectly work as same as principled bsdf,, with keep each socket.. (for clear coat) ,,

so you can merge it in custom shader refraction group (for principled option).

(base = principled bsdf without refraction (for non reflective part)

(2nd = custom refraction shader, with clear coat, or your custom top coat )

then base and 2nd are mix with refraction weight.

Then we know there should be case,, iray use cut-out opacity,, about this case,,

we need to plug cut-out opacity map,, for final mixing with pure transparent shader,because I suppose we often forget to make alpha node, for 2nd shader.

‌

‌

[engetudouiti]

To work simple I recommend make perfect glass shader for eevee and cycles spearetly, then about 2 option, use it. (base = principle or full bsdf)

after all we may need 2 shader group for each option (eevee and cycles)

so even though we make one glass shader (include thin wall switch) carefully (of course it is ok divde as 2 group for cycles and eevee) no user may complain, because you can see most reliable shader with both option.. and it make work more small.

we only need to test the multi glass shader for refraction part. first..

then everytime for all mat,, we may add it when there is refractive surface. it can be base to make things stable because iray actually use refraction weight so.. do not think so? then if for refraction you hope to add volume,shaders you can add it later.. (for cycles only I suppose)

(then finally cut-out opacity will be added just for remove the part perfectly)

then make eevee glass shader (with thin wall or without thin wall ) is main project ^^; If it is already done,, there is no problem.. just mix with by refraction map for each base..

[Alessandro Padovani]

Xin, I agree with you as for the refraction map, since it was missed I bolded my comment above.

[Alessandro Padovani]

Xin, on a second thought, in iray the diffuse color is ignored for refractive materials, so if Thomas wants to avoid the double principled to keep it simple, then the actual solution is not too bad. I mean, it points to the user that something is wrong because this is visible in the eyes, but it also leaves the refraction map if the user wants to deal with it.

In my case for example I get the wrong V7 eyes, then I delete the refraction map and I’m fine. While you may add a second principled node.

Since the principled option is supposed to be approximated for complex materials, and it is expected that the user may need to fix it, this may also be an acceptable compromise. For good refraction handling complex cases, the bsdf option is there. I believe this is what Thomas means in his reply. So I’m also closing #333.

[engetudouiti]

of course every shader may not use diffuse color for refractive part. but use base color (principled bsdf transmission), or use refraction color or use transparent color.

So , refracton weight map must need to decide it and force shader to use which map will be used as base color for principled bsdf.

of course about some material if you need not clear divide it, and if it work it may OK, but if you get iray glass mix shader preset, how you conver it for cycles?

if we really really really use one principled shader only, for non refractive part and refractive part,, we need to use mix color node, then plug refraction weight as factor, then change with condition,, so we can use same input (base color) for 2 purpose.

(one is for non refractive part, 2nd is for refractive part)

it often need to uber type shader.. but plug in may not do it .

‌

‌

‌

[Thomas Larsson]

I actually had to work today (I only work two days a week thanks to covid) and didn’t have time to respond before. I understand the logic of making the opaque and refractive trees separately and then mixing them together, but the code wasn’t originally written that way and implementing it would be a major revision. Instead the global options have changed again:

There is a single material method which defines the type of node tree: BSDF or principled.

If the principled method is chosen (and only then), there is a second option which defines how the refractive part of materials are handled:

• BSDF: Add a refractive node group.
• Principled: Add a second principled node.
• Squeeze: Use the same principled node as for the opaque part.

The two first methods work reasonably well, at least for Vic7 eyes, and the squeeze method has problems. But with such an unappealing name, the user should expect that.

‌

[Alessandro Padovani]

Thomas, I’d suggest BSDF + Principled-PLUS + Principled as names, since squeeze is a big guess as for its meaning. And may be include #333 in principled-plus if I understand correctly what the new options are for. Please let me know what you think.

[engetudouiti]

I know I only show aproximation. (basically refraction weight are 1 or 0,) but there is no rule , actually I often see mid color used for refraction weight.

when there is middle value (0.3 0.25 etc), my simple color mix way not show same final (r,g,b) for those point.

A diffuse(r, g, b) is tex(r g b) diffuse effect. B refraction (R,G,B) is tex(R,G,B) refraction effect,

real (use 2 nodes ) =

refractive shader (r, g, b) * w + main (R, G, B) * (1-w)

aploximate way (only 1 nodes) =

refractive shader( (r, g, b) *w ) * w + main ((R,G,B) * (1-w)) * (1 -w)

it is what I did. (so it only work, w = 0 or 1)

btw, I think recent up-date options are enough for me it is clear to understand how add on will generate.

[Thomas Larsson]

Changed the name from squeeze to none, since the other names tell what node is added. But the result for Vic7 eyes are not good, even when mixing the textures, so this option should probably be removed altogether.

[engetudouiti]

It do not need anymore but, actually I miss circulate. real is a little bit more compex.

single node + mix RGB (with refraction weight ) do

refractive-shader( (r,g,b) w + (R,G,B) (1-w)) * w +

main shader ((r,g,b) w + (R,G,B)(1-w)) * (1 -w)

it still show same final color when w = 0 or 1. (so pure black and white image can work, but where gray used,, it show different color from daz do

Then we can not import These product mat, with None option well.

‌