Option to read HD morphs formulas and create a driven Value node in a material to replicate the driven HD morphs with driven baked normal maps

[Xin]

You can’t read HD morphs from the files, but you can read their formulas, so it would be quite useful for people who choose to bake normal maps or displacement maps of HD morphs at full effect to get some of the automatic handling which driven Shape Keys would provide. The idea would be for the addon to add the possibility to create a driven Value node in the Shader editor whose Driven expression replicates the formula from the HD morph (that is, it goes from 0 to 1 as if it was driving a Shape Key).

This driven Value node could be used, for example, to “mix in” the baked normal maps of the HD morphs with the base normal maps automatically, effectively getting some of the driven HD morph effect back through normal maps (although the node could be used to mix other things too, like displacement maps).

[Thomas Larsson]

There are several ways to do this, and it is not clear to me how to do it best, if at all.

1. One could add a separate button that adds such a value node to selected materials, driven by a custom morphs property. This should work for HD expressions, but not for HD JCMs (if there is such a thing), because the driving functions are more complex that just a linear function.
2. One could read the formulas when importing the morph and add value nodes to all materials. Or perhaps only for HD morphs. If you import a HD expression which only affects the face material, you will still get nodes in the other 16 materials too, which seems like a waste.
3. Can you add normal maps? I tried with a vector math node, but the result was not good. Your idea only seems useful if you can use drivers to mix several normal maps, and you would need some kind of node group for that.

‌

[Xin]

My motivation was to get some support to emulate tension maps (these are driven by geometry itself, not bones, so they are quite heavier): https://blenderartists.org/t/revised-mesh-tension-add-on/1239091 (scroll down there to see different typical shader setups used with that method to combine normal maps).

You can’t add normal maps, but you can either mix them, or you can overlay them, although the latter is non trivial: https://blog.selfshadow.com/publications/blending-in-detail/ . I attached a .blend which includes a node that implements the overlaying described in that article (credits to Creative Shrimp). But I don’t think the addon should even bother about this aspect, the mixing should be up to the user. Maybe create the “combine normals” group node from the attached file as a convenience node (maybe as an option when telling the addon to create the Value/Driver?). But I think that the concern of the addon should be defining the Value/Driver since it’s a job it already does very well. Anything else varies too much from case to case for the addon to decide for the user.

I’m not sure I follow you when you imply that the Value can’t be driven by bones like JCMs. You can make a Driver for the Value node and include bones' poses in its expression, I tried that and it worked. I even copied one of the generated driver expressions created by this addon for one of the JCM shape keys and it still worked. What am I missing? is something specific about the custom python evaluation function you use? scaling/clamping issues can be resolved by the user if needed.

As for the UI, I don’t think anything automatic would be good. As you say, it’s not a good idea to add too much stuff to materials unless it’s needed. So I was thinking of a button which reads the selected material and creates the Value node/Driver on demand (after indicating which morph to get the formula from). Maybe more suitable for the advanced section.

[Thomas Larsson]

There is now a button that Add Driven Value Nodes. It is located at the bottom of the Advanced Setup > Mesh section. Not sure if that is the right place for it.

Normal maps are now mixed with an Overlay node. It may not be theoretically correct, but it is simple and a lot of people seem to use it.

[Thomas Larsson]

The materials section was becoming too big, so I split it in two, and put rarely used buttons under the Advanced Setup panel. The Add Driven Value Nodes button is moved there, together with the buttons that bake and load the maps. I also added one more button, Add Normal Maps. The idea is this:

1. Load the base normal maps to all materials.
2. Add expression normal maps (texture and overlay) to the face material.
3. Add driven value nodes to the face material.
4. Connect the driven values to the overlay Fac inputs.

‌

[Alessandro Padovani]

This sounds great. I hope to see a easy tutorial in the blog about how to use it in a practical case, for those of us that are not rig experts. So do we now have HD morphs and jcms via displacement maps ?

[Thomas Larsson]

So far only normal maps are considered, but it shouldn’t be hard to extend it to height maps, you must just decide how to add them. With a math node set to add, perhaps. There will still be a lot of work, because the normal maps for morphs must be baked one by one. OTOH, you can probably get away with reusing them for other characters.

‌

[Alessandro Padovani]

Well normal maps are good because they work both with cycles and eevee. On the other side displacement maps are probably what makes more sense to get true HD details, especially for jcms driving muscles, but they don’t work with eevee. In any case this sounds as a great step forward to HD support.

Also I'm not sure to get if/how we can drive the HD morphs directly on the HD mesh, as suggested by Xin in #300.

[Xin]

Oh nice, thanks Thomas. I will test it as soon as I find time.

Alessandro, I don’t understand what you refer to, could you quote the comment?

[Alessandro Padovani]

Xin said: “With the rigged non-multires mesh you can bake the HD morphs from it to either the multires HD or to a non-HD mesh, both of which can’t load HD morphs.”

From your comments in #300 I understand it’s possible to transfer the rig to the HD mesh, then load the HD morphs there. Then from your comments here I understand it’s possible to drive the HD morphs. If I don’t get it wrong.

Then from Thomas I understand this would be very slow and memory consuming. But since HD figures are not intended for animation I believe this would be acceptable in this case.

[Xin]

Trying to automatically set up the drivers with the addon I think found an issue (not directly related to this issue).

To make it simple, try to import the following eCTRLs and eJCM for V7 and see if you can get the drivers set up (just the regular drivers, not the ones introduced in this issue for the node editor):

• Standard expression Fear:
  “eCTRLFear.dsf” in “…\data\DAZ 3D\Genesis 3\Female\Morphs\DAZ 3D\Expressions”.
  This one imports well, just mentioning because the ones below depend on it.
• “eCTRLVictoria7Fear.dsf” in “…\data\DAZ 3D\Genesis 3\Female\Morphs\DAZ 3D\Victoria 7”.
• “eJCMVictoria7Fear.dsf” in “…\data\DAZ 3D\Genesis 3\Female\Morphs\DAZ 3D\Victoria 7”.

---

I can import both the “eCTRLVictoria7Fear” and the “eJCMVictoria7Fear” as custom morphs but I couldn’t manage to get them imported with the drivers set up.

Notice that “FHMVictoria7.dsf#FBMVictoria7_v14_4_6_6” will get imported in the process. I assume it’s because “eCTRLVictoria7Fear” is multiplied by it in its formula, but I don’t know if this is intended behavior.

[Xin]

Alessandro, yes it’s possible to transfer the rig to the real HD mesh quite fast by using this addon to import the rigged HD with multres, and Blender’s Data Transfer modifier to transfer vertex weights from that multires HD to the real HD (Nearest Vertex matching works quite fast, less than 10 seconds with 250k vertices). Thomas has recently implemented a vertex weights transfer function in the addon too, but I haven’t tested it yet.

After doing that transfer, you end up with the rigged real HD as if you had imported the real HD with daz’s Blender bridge. The process I described was intended as an alternative to daz’s bridge. So, in principle, you can import the HD morphs to it manually (that is, as .obj) and you have the working rig too.

The following process would also be possible:

• Transfer the rig to the real HD mesh as explained above.
• Import all the HD morphs (and pJCMs) you want as .obj to that rigged real HD. This addon could then set up the drivers by reading the formulas from the files (these, unlike the actual vertices' offsets, are public). This can’t be done right now but, in principle, it wouldn’t be that hard for Thomas to implement it (POINT A).

But I don’t consider that worth the effort. At this point, just bake maps. Normals and displacements are animation friendly, that’s why I suggested drivers in the node editor in this Issue. Such a process could be:

• Import the real HD (either manually, or by disabling the multires in this addon).
• Import all the HD morphs you want as .obj to that real HD. They will be raw Shape Keys ( POINT B ).
• Dial the HD Shape Keys one by one, and bake normals from the HD mesh to the base mesh to get normal maps.
• Now you can use the feature requested with this Issue to automatically drive those normal maps with the poses of the bones of the base mesh’s armature, similarly to how the HD morphs are driven by the rig in the real HD mesh inside daz.

An alternative to save some manual work at the cost of really long import times:

• Import the real HD with daz’s bridge and add all the morphs you want in the dialogue it presents you. The Expressions and eJCMs will be exported as raw Shape Keys, they won’t drive bones like they do in daz, they will just deform the mesh directly (this happens even with the base mesh, yet another reason why this addon is better). Another issue is that the HD pJCMs seem to be completely ignored, so you will have to import those manually as .obj.
• The bridge will convert Expressions and HD eJCMs to raw Shape Keys (as opposed to posing bones too as they do in daz) and will ignore the HD pJCMs. So I don’t see much of an advantage here relative to POINT A. But if you just wanted to bake normals (as an alternative to POINT B ), this method would save you the trouble of having to import each morph as an obj separately (provided they showed up in the dialogue that daz presents you). They will all be imported at once (the import time will be really long though). You can then proceed like after POINT B (but beware to ONLY turn on Shape Keys one by one or the bake will be wrong).

Yet another alternative thanks to Jochen Sutter which I like, and which also allows you to bake displacement maps (Blender 2.8+ can’t bake displacements except through a multires, unlike Blender 2.79) is:

• Turn on the morph you want to bake to the max inside daz before exporting the mesh with this addon’s script. Then import it with multires to Blender.
• Since the vertices' positions are “baked” in the .dbz file, once you import the mesh in Blender it will have the real HD vertices positions as if the HD morph was active. You can then use the Bake Normals/Displacement button from this addon to get normals/displacements quite quickly (no need to have a base mesh lying around, the multires already has all the info it needs).
• Then you can proceed to drive those baked normals/displacements with the feature introduced through this Issue. Perhaps you could even use the baked displacement map to create actual displacement in the multires mesh in Cycles, effectively getting a similar automatic behavior as if instead of a driven displacement map you had a driven HD Shape Key. Again, I don’t consider this worth it, you might as well bake normals and use them on the base mesh, and you will get an HD look that is also animation friendly.

‌

[Alessandro Padovani]

Thank you Xin for the nice and detailed explanation of the various options we get. I believe this can be a nice addon to the docs.

Personally I’m not a big fan of HD figures and I agree with you that baking normal maps is better for animation. And that’s also the original idea for HD to be imported. But I know there’s some daz people who can’t live without true HD geometry, so if any improvement can be done for POINT A I’m quite sure it will be welcome, even at the cost of memory and speed. If Thomas agrees.

[Thomas Larsson]

You can import the drivers now. More precisely, if you import a HD morph (which references a dhdm file), and the vertex counts do not match, a shapekey is created nevertheless. You can then get the driver from the shapekey value. This works when importing custom morphs, jcms, and flexions.

If you have a separate HD mesh with the morph applied, you can transfer the locations to shapekey just created. I’m not sure if there is a standard way to do this, so I made a new tool for it. Just be sure that there are no object transforms when you do it.

[Alessandro Padovani]

There’s a tool to do it but it requires some extra steps.

1. join the HD morph as shapekey (join as shape), so we now get a new shapekey with the HD morph
2. enter edit mode
3. blend the HD morph to the desired shapekey (vertex menu > blend from shape), so we copy the HD morph to the desired shapekey

‌

If I understand correctly, this seems complete enough as a first implementation of the idea. It requires some manual work but the tools are there. May be a quick tutorial in the blog with a practical example will help to better understand and use the new tools.

Thank you so much Xin and Thomas for these new nice features for HD figures.

[Thomas Larsson]

I added a third option: Instead of creating an empty shapekey with driver and fill it in later, you can also import the drivers to the active, existing shapekey. If you want to create the shapekey first.

[Xin]

Thanks Thomas, those options will be useful.

I have tried to set up eJCMs to test this feature (as opposed to using pJCMs since I was testing face details), and I haven’t been able to make the addon build the drivers completely. Here is what happens when I try to import these:

• Standard expression Fear (imported through the Expressions button):
  “eCTRLFear.dsf” in “…\data\DAZ 3D\Genesis 3\Female\Morphs\DAZ 3D\Expressions”.
• “eCTRLVictoria7Fear.dsf” in “…\data\DAZ 3D\Genesis 3\Female\Morphs\DAZ 3D\Victoria 7”.
• “eJCMVictoria7Fear.dsf” in “…\data\DAZ 3D\Genesis 3\Female\Morphs\DAZ 3D\Victoria 7”.

The yellow lines go from driver to driven (the eCTRLFear is the same as Fear). The green lines indicate the drivers the addon isn’t creating which exist in daz. Am I misunderstanding something about how the addon does this? I don’t know if this is intended for a reason, I don’t use dialed Expressions much.

Here is an example with the drivers manually set up:

Maybe this should be an option if it doesn’t exist yet? I could see why you would still prefer to keep them all drivers-free to have more control (the current behavior). But deleting a driver is always easier than creating it.

Also, I think eJCMs in particular should resemble pJCMs, in that they should have an option to be imported and be driven by the relevant rig properties if they exist (not just bone poses). That is, an option to allow them to be automatically driven by the properties/bone poses from its formula (in this case just a single property: eCTRLFear). Right now, if you try to import the eJCM… as a Custom JCM, no driver is created, just a raw shape key.

In the example above, eCTRLFear could directly drive the Shape Key eJCM…; then you could remove the (quite useless) rig property eJCM… of the same name.

As a side note, I think all rig properties that are just aliases for Shape Keys should be grouped together to differentiate them from useful rig properties like the eCTRL ones, which don’t exist anywhere else.