about the morph database

Issue #1106 resolved
Alessandro Padovani created an issue

The blog news sound amazing if we can now simplify morphs. Here are my impression and notes if it can help.

1. To scan the database for G8F G81F only took about one minute on my system (ryzen 2200, bx500 ssd). I keep in the daz content folder only the assets necessary for the current project, when I change the project I switch the content folder. So working this way makes things fast and easy both in daz and blender.

2. The scan database is not available on a empty scene, we have to load a daz figure first. Since it's independent from figures it may be better available in the global settings. Also it may be useful if we can choose the database location and reset it if needed. For example in my case I'd store the database with the project and switch it when I change the project.

3. Here the imported morphs don't seem to work fine. At least as explained in the blog. Below an example with BEF06 as it is in daz studio and blender. Perhaps the div2 is not used ?

steps:

  1. import G81F and merge rigs
  2. scan the daz database for G8F and G81F
  3. import FACS with make all bones posable
  4. import BEF06 with import morph from pose and autokey enabled

Comments (68)

  1. Alessandro Padovani reporter

    p.s. BEF06 works fine if I load the custom morph instead of the morph from pose.

  2. engetudouiti

    Should not name it as Preset data base? I sometimes feel, Thomas call Preset file as morph..

    On the other hand, for ds say morph file , it must need to have slider (controller) and they should be installed in data directlry.. (though there is case, which we temporally generate slider and save it as scene >>> it make daz scene really large, and the morph only work for the scene.

    At current I do not clear understand, what Thomas call morph… I suppose it can not import my custom morph.. but may import my custom preset (which may mix use other morph, but it just set value , how mix each real morph)

  3. engetudouiti

    I do not test with G8 but test and import some expression = preset file.

    Then I notice, it change already imported morph value. at same time,, I do not know the difference when we save preset, and load…

    One thing I may expect, so even though there is miss controller, which I had not installed by daz importer, it may circulate ERC and apply? at same time, if the controller need new geometry data, it may not work right?

    eg, even though I try to import G3 FACT expression, it never work. (I test it), because it need to set pose controller… it is usuall,

    But if I import expression preset which use my custom morph(aka shape key data) with pose bones,, but I have not installed my custom morph in blender, how it work?

    Or new expression importer can only manage… Facial bone pose controller used preset? that means, the preset may only drive,, bone related controller. but never add new morph.

    (or only use base face unit morph = which we should install it)

    Or one expression file = TestExp, may drive Face unit + Expression category morph. then user did not install Expression morph, but only install Face unit.

    about the case, new morph preset can install the TestExp and drive correctly? I feel it depend, eacn base morph include mesh delta data or not.. if one of related morph include mesh-delta. I feel,, new import system can not manage it… or it need to add new shape key after all…

  4. engetudouiti

    But I understand maybe it expand to import expression preset ..(mainly which may only pose bone + other pose controller which not include delta-morphs, or only change morph which already imported)

    Though I do not understand, when it may not work clear… it seems depend how the expression preset was made I feel… if some of related ERC morph include verts delta infomation, these morph should be improted first, I think so.. Then maybe most of G8 product only use, pose bone, and Fact morph, I suppose… so it may work..

    On the other hand, good G3 expression vendor product, often add new delta shape slider and mix use daz default expression.. (and sometimes, mix use the vendor add new morphs) >> it may not import well I suppose. because it need to add delta verts as same as jcm.

  5. engetudouiti

    Maybe if we import related morph correctly, use import pose with activate effect morph show more reliable effect for me.

    A Import pose with effect morph (which already imported) option. (old way, I seldom use it though)

    about same figure, just import same expression by new import morph system

    with some test I clear see, usually upper (around eye) pose seems often weaken hardly. when use new import morph system.

    (I suppose if morph include bone scale,, after all it not work though, I found many scale value in dis pose duf (i11 expresison product).

    Most of i11 only use, basic facial morphs + directly manipulate facial bones as delta add. so it may work.. (I have already imported as face unit, expression)

  6. engetudouiti

    Of course I know it is beta, (and suppose most of user now move to G8.1 , at same time about morph I content with current status, so I may sometimes test if it can enhance my current work-flow.

    (though hope to see some enhancement to manage face bone scale well, etc, (but I feel it really complex, even though I only test G3 with new ERC morph system)

    btw my best recommend way is manuall work … after all just make expression which only use bone posing + basic shape morphs (only when it needed), then import as pose == usually work well without heavy perfomance issue.

    So I often convert daz vendor expression preset with Bake ERC >> then convert them which only use bone posing. (remove intermediate ERC, and bake as pose directly), then save it, and load it with daz importer. (yes it take long time, but once convert it, it work best)

  7. Alessandro Padovani reporter

    You can also export the fbx from daz then transfer the fbx shapekeys to the diffeomorphic figure that’s probably faster than freeze erc and bake to transform. That is, you do in blender what you’d do in daz studio. See #43.

    What Thomas does here is another thing though. You don’t import the extra morphs at all but only use the basic morphs. I guess the database is searched for the used morphs in the pose then the pose expression is evaluated and the result is baked into morph keyframes.

    ex. bef01 = fac1 + fac2 + fac3

    note. Please note the big expressive are not simple presets, otherwise they’d work just loading the pose. The PA probably first created the presets, then erc freezed them to new morphs, then created the poses for the new erc freezed morphs. This Thomas is trying to reverse with the new morph from pose. Indeed the daz erc freezing is bad for blender because it creates an extra layer of morphs to drive other morphs.

    https://www.daz3d.com/big-expressive-81-for-genesis-81-female

  8. engetudouiti

    About FBX it is perfectly different meaning,Because now I talk about how to use Daz Expression preset file, then Just import and apply bone transform. with check each file pic. temporally or set key.

    Then I think yes, Thomas do so. then I ask, if it can apply mesh delta deform or not.. it is not pose weight deform but directly move verts as Morph.

    If it can only circulate Bone pose ERC and bake it as 100 strength, it is almost same what I do manually in daz studio. The difference is, Thomas way is far more quick. so if it work well I expect much. But DAZ bake transform need not any Basic morph, because it can convert all ERC reation as Pure Pose bone transform only. (about this point, I think it is best way to import expression without any driver issue. because it need not to import basic morph = need not drive basic controller.

    Then if this tool will be stable, (catch all ERC and apply it correctly) I may expect, (and had requested), if Thomas can make “Bake Pose Transform” = Convert many complex ERC (which only pose bone related) as pure bone transform value for blender. (then if it work, I do not think we need to use basic morph even though it had been imported.

    But about G8 or G8.1, it maybe more complex. so not expect much. maybe It is more useful to use basic morphs + bake other pose bone controller as bone transform.

  9. engetudouiti

    Then I never say, to make such tool is simple.. I know clear about how daz make complex ERC. But actually “Expression file” = preset is simple.. there is no big or small thing.

    All Pose or Expression preset usually contain only simple data. key and value;. they just describe how set Each morph value. which had been installed.

    the complex part is how to circulate each morph file and covnert as Bone pose. If it only use basic morphs it is ok, but at same time it means, it should lost other morph deform which user had not installed. like some MCMheadmorphs. or vendor added Corrective morphs for head expression.

    So I may ask, what expressoin can be imported. or not. and I hope to know when it will be failed. from Thomas. or I simply try to import any expression. (though actually I guess, which expression may be imported correctly, but which vendor expression may be failed)

  10. Alessandro Padovani reporter

    update. Commit fac868f

    I see some work is done on the database. So just to update things, bef06 stills not working as reported in point 3 of the first post. I also scanned the standard morphs. As for point 2 I see now we can choose the database location, but still we can’t scan if we don’t import a daz figure first.

  11. Thomas Larsson repo owner

    Some of the issues are adressed in the latest commits.

    The Scan Morph Database button is moved to the top level, below the Global Settings button. It couldn’t remain the the Morphs section since the entire section is disabled unless an armature or a mesh is active.

    The tool missed some morphed before because aliases were ignored. Aliases work well if the base morphs have already been loaded, because then the real morph name corresponds to a custom property, but if they are not loaded we don’t have the definition of aliases either. More work is needed in this area.

    I agree with engetu that the word “morphs” is used in too many places. The purpose is to load a pose preset, located in the Expressions folder, which contains references to the Morphs data folder.

    If a file contains a combination of morphs and poses (including ERC), you should be able to import both by selecting the same file with both the Import Pose and Import Morphs tools. I think, but I haven’t yet seen an example.

  12. engetudouiti

    Yes though I may not hope to against detail but, I think your new tool is “Load expression (preset)”

    then about daz, it is one of Pose preset (include any morph controller and bone pose). but only about Expression (aka head pose)

    But I understand clear your aim. (and I understand it should show better result, when we import all base morphs (which include shape deform morph controller (shape key)) than my bake as transform way (daz offer tool)

    because, if I convert all deform as pose bone deform, it ignore any shape key deform which include in basic morphs.

    So your new way is,, bake all other morph cotnroller effect as Base morphs + bone transform, I suppose.

    (then most of expression preset file should work, if you finish it, without import other controller, add on may check them and bake it as basic morph + bone pose)

  13. engetudouiti

    Real vendor famous product for G3,, i13 100 esseintial expression for v7 it actually include some facial bone scale + base morph vallue (mix use, vismes + face unit + expression)

    one of example source. (i13 may not blame me even though I show it,,😅 ) (to aovid this, I usualy made custom pose,, but you may not believe real vendor use it, without I show example)

    "scene" : {
            "animations" : [
                {
                    "url" : "name://@selection/upperTeeth:?translation/x/value",
                    "keys" : [ [ 0, 0 ] ]
                },
                {
                    "url" : "name://@selection/upperTeeth:?translation/y/value",
                    "keys" : [ [ 0, 0 ] ]
                },
                {
                    "url" : "name://@selection/upperTeeth:?translation/z/value",
                    "keys" : [ [ 0, 0 ] ]
                },
                {
                    "url" : "name://@selection/upperTeeth:?rotation/x/value",
                    "keys" : [ [ 0, 0 ] ]
                },
                {
                    "url" : "name://@selection/upperTeeth:?rotation/y/value",
                    "keys" : [ [ 0, 0 ] ]
                },
                {
                    "url" : "name://@selection/upperTeeth:?rotation/z/value",
                    "keys" : [ [ 0, 0 ] ]
                },
                {
                    "url" : "name://@selection/upperTeeth:?scale/general/value",
                    "keys" : [ [ 0, 1 ] ]
                },
                {
                    "url" : "name://@selection/upperTeeth:?scale/x/value",
                    "keys" : [ [ 0, 1 ] ]
                },
                {
                    "url" : "name://@selection/upperTeeth:?scale/y/value",
                    "keys" : [ [ 0, 1 ] ]
                },
                {
                    "url" : "name://@selection/upperTeeth:?scale/z/value",
                    "keys" : [ [ 0, 1 ] ]
                },
                {
                    "url" : "name://@selection/tongue04:?translation/x/value",
                    "keys" : [ [ 0, 0 ] ]
                },
                {
                    "url" : "name://@selection/tongue04:?translation/y/value",
                    "keys" : [ [ 0, 0 ] ]
                },
                {
                    "url" : "name://@selection/tongue04:?translation/z/value",
                    "keys" : [ [ 0, 0 ] ]
                },
                {
                    "url" : "name://@selection/tongue04:?rotation/x/value",
                    "keys" : [ [ 0, 0 ] ]
                },
                {
                    "url" : "name://@selection/tongue04:?rotation/y/value",
                    "keys" : [ [ 0, 0 ] ]
                },
                {
                    "url" : "name://@selection/tongue04:?rotation/z/value",
                    "keys" : [ [ 0, 0 ] ]
                },
                {
                    "url" : "name://@selection/tongue04:?scale/general/value",
                    "keys" : [ [ 0, 1 ] ]
                },
                {
                    "url" : "name://@selection/tongue04:?scale/x/value",
                    "keys" : [ [ 0, 1 ] ]
                },
                {
                    "url" : "name://@selection/tongue04:?scale/y/value",
                    "keys" : [ [ 0, 1 ] ]
                },
                {
                    "url" : "name://@selection/tongue04:?scale/z/value",
                    "keys" : [ [ 0, 1 ] ]
                },
    

    only one file include at least 10 or more scale value, with use other basic morphs. about this product.

    (so you know, to make expression preset, there are many ways, then each vendor make them as they like. at least it can load and deform correctly in daz studio ^^;)

    about tongue, I may not hope to touch scale though.. but same file include other facial bone scale.

    {
                    "url" : "name://@selection/rLipLowerOuter:?scale/general/value",
                    "keys" : [ [ 0, 1 ] ]
                },
                {
                    "url" : "name://@selection/rLipLowerOuter:?scale/x/value",
                    "keys" : [ [ 0, 1 ] ]
                },
                {
                    "url" : "name://@selection/rLipLowerOuter:?scale/y/value",
                    "keys" : [ [ 0, 1 ] ]
                },
                {
                    "url" : "name://@selection/rLipLowerOuter:?scale/z/value",
                    "keys" : [ [ 0, 1 ] ]
                },
    

    I feel interesting, because, there seems 2 case, even though vendor use general scale. (some duf, seems show general + each x y z scale value, but when I make it, it not…so hope your add on not make double count… I hope,,

    about his product, he seems use bone translate ,rotate, when it need. about scale keep 1.0 usually used to return bone scale I suppose . use translate example .

    {
                    "url" : "name://@selection/rBrowMid:?translation/y/value",
                    "keys" : [ [ 0, 0.2326256 ] ]
                },
    

    rotation example

    {
                    "url" : "name://@selection/lBrowOuter:?rotation/z/value",
                    "keys" : [ [ 0, 15 ] ]
                },
    

    I can usually find those, for most of i13 expression file. (some vendor only use pose bone transform to make expresison/pose preest, I remember, and I prefer his way,, only use other controller when it need)

  14. engetudouiti

    about general scale,, I think it need to multiple with each x, y, z scale, when it was described.

    my added i11 example was bad, because it almost keep 1.00 about scale, just describe in preset about scale make his work simple..

    then

    if there is only general scale described,, eg 1.5 you need to set x, y, z as 1.5 (you already done)

    if both are described, general scale need to multiple with each (x, y, z) , because in daz when I tweak general scale, it not change x, y, z scale value.

    . (so there may be case, we double use it, eg general = 2.0 and (1.5, 1, 1) for each. in blender need to convert as x = 3.0 y = 2.0 z = 2.0

    though I remember we talked about same thing before, but just confirmed. (and believe you made it so already, then now general scale only described case already worked)

  15. Alessandro Padovani reporter

    update. Commit c4f06c7 still doesn’t work for bef06 as reported in the first post.

    As for poses vs morphs I agree it’s a pose we import. Actually we have “import pose“ that loads the pose, and eventually also loads the missing morphs. Then we have “import morph“ that loads the pose, but instead of loading the missing morphs it bakes them into the pose by reading the morph database, so to only use the base morphs.

    May be instead of having two buttons “import pose” and “import morph”, we may add an option to “import pose“.

    missing morphs:

    • ignore
    • report
    • load face
    • load all
    • bake from database: only use base morphs, and bake intermediate morphs into the pose by reading the morph database

    note. Then in my opinion “bake from database“ should be the default since this makes most sense in blender, where we don’t want morphs driving other morphs as in daz studio.

  16. Thomas Larsson repo owner

    engetudouiti, I don’t expect to include bone poses in general; the Load Pose tool already does that. However, I did add an option to handle the common case that the pose preset clear some bones, presumably the face bones. Typically we may load a body pose with Load Pose and an expression with Load Morph, and in that case we only want to clear the face bones.

    I tested with the BEF morphs. For some reason the PA chose to clear the face bone rotations but not the translations. Weird, but that is what’s in the duf file.

  17. Thomas Larsson repo owner

    Alessandro, what you say makes sense logically, but for my own workflow it is useful to have two different buttons. I typically first load a body pose and then an expression, and it is very easy to forget to change the tool settings each time you switch. But perhaps that could be done with two copies of essentially the same button. Anyway, it should not be too hard to let Load Pose use the scanned database too.

  18. Thomas Larsson repo owner

    OK, now I understand the problem with BEF06. It also needs a custom shapekey, but the scanning code ignores such files.

  19. engetudouiti

    Thomas I do not expect to load “body pose” with import morph (though if it work for all expression, I might hope to import some breast pose pack, (which may use vendor offer pose controller and set the value)

    What I means,, eg i13 expression products include, Facial bone pose + basic morph value ,. then

    at current I think add on failed to load them as Import morph.

    about this case, which function you recommend to load Expression?

    (So I said, Expression is just naming way for daz,, when it change face as pose, they call it as Expression)

    eg I make expression preset like this. (maybe many user do so)

    1, use basic facial pose morphs related with face (head pose controller section)

    2. if there is vendor special product which can use user may add it too.

    3 finally fine tune by pose each tiny facial bone, to adjust detail to fit current character.

    4, save them as pose preset>> just save in Expression directory. (or user can free decide location)

    Then now I only import base morphs (facial unit, expression, vismes morphs) in blender.

    About this case, I expect “import morph” in pose section can load my “expression preset” without load other product controller.

    so if it can manage this type, it should load only facial bone pose which save as expression, (if it work, I may use import pose for Body pose preset, import expression (aka facial pose preset) by import morph.. I suppose it is what you means…

  20. engetudouiti

    And I really need to say, we may not expect daz naming about those preset file have actuall meaning..

    it is just used to make group to user load easy . but all preset file actually do same thing.

    they can use any controller (almost all), then you can use it as Pose/shape/parameter as you need..

    So any pose preset can include, any shape preset, if vendor or user make so.

    and any parameter can change type as shape morph/ as pose morph etc ^^;

    So basically even though you try to devide, the file strutucre, (it only include pose controller, shape controller or bone transform), they easy mix up in one preset file.

    ===

    Then if

    import pose = it only change body related morph + body bone pose

    import expression = it only change face related morph (or bake it for standard face related morph + facial bone pose)

    it make sense.. (but at current, at least most of G3 may only need to use import pose,, because most of expression file actually only use G3 base face related morph + bone transform only.

    so if user have imported G3 base face related morph, (aka face unit- expression- vismes, some head shape morph it is used by facial pose morph), at least most of G3 expression can load it. (they do not use other morphs basically, but need to manage bone transform )

  21. engetudouiti

    “I understand the problem with BEF06. It also needs a custom shapekey, but the scanning code ignores such files.”

    then actually it is what I asked. , I afraid it.. (so hope you confirm work or not)

    “““But if I import expression preset which use my custom morph(aka shape key data) with pose bones,, but I have not installed my custom morph in blender, how it work?”””

  22. Alessandro Padovani reporter

    @Thomas Well if “import morph” is supposed to be used for expressions then it could be named “import expression“ to be clear. This also makes sense because we have a separate folder in daz studio for expressions. Though technically there’s no difference between poses and expressions, workflow-wise it may be convenient to get them separated.

    As for bef06 yes, as I wrote in my second post it works fine if I load it as custom morph. Then if “import morph“ can’t handle this case I fear it makes little sense and we're back to using custom morphs. I mean the morph database could also be used to load the missing custom morphs when we load an expression, the same as “import pose“ does.

    That said, if this is an intended limitation for “import morph“ it should be explained in the blog. Then if @Enge has nothing more to add I may also mark as resolved. Let me know.

  23. engetudouiti

    @Alessandro it is OK free close it.

    I keep asking about this new sytem in my topic when I need. and I agree (from start point I said so!!!), Thomas may better to change function name. (it not related it will bake as base morph etc) to import as Expression (aka facial pose or pose preset for head part) = then at same time If you change it, you may consider pure facial bone pose preset. because it is actually Expression file. when daz user categorize.

    Then as my request, why it only work when we save the default location? (or it work where we save expression file? )

    many daz old user, free save their preset file (expression preset) as they like , so it may change loation easy.. ^^; (then only use default expresison location for Vendor or product)

    if new tool is for vendor expression only, it make sense though… (yes I suppose so, yours seems not try to use custom preset etc, it is pity for me,, you should use your custom preset more.. (and custom controller with mix use pro offer product >> enhance it as you like >> import for blender

  24. Thomas Larsson repo owner

    Puh! Now the scanned database has been fully integrated with the other pose-loading tools. There are now two separate buttons: Import Pose and Import Expression. Both load pose presets and are essentially the same, but the default settings are different, although you can change the settings so both tools import both poses and morphs. The names indicate that you typically would load a pose from a directory called Poses, and an expression from a directory called Expressions.

    The scanned database now also contains morph files that are shapekeys and not just formulas. Therefore the BEF06 shapekey is loaded.

    On a side note I think that the PA was a bit careless with his naming; the main BEF morphs are called “01”, “02”, etc. This works since there are no other morphs with the same names in my database, but this is certainly asking for name collisions.

  25. engetudouiti

    Thanks now it seems clear than before.. . then so can you confirm these?

    1 I make pose controller for daz correctly (it may use other vendor expression morphs, and my customize pJcm for the controller, (which already imported in daz data) + adjust tiny facial bone

    then save the expression (save as pose preset) in my custom directory (in DAZ library with my specific directory)

    2 I load all basic morphs (face unit, vismes, expression).

    3 import my saved expression by import expression..

    Will it work? if not work what is problem?

  26. Alessandro Padovani reporter

    As for commit 7daa08b I’m confused.

    Now if I load bef06 a custom morph is loaded and bef06 works fine. But if I load bef01 a custom morph is loaded for bef01 as well, that didn’t require a custom morph before and seemed to work fine as “baked”. Is it that all the bef expressions really require the custom morphs, or is this a bug ?

    Also there’s some weirdness because bef01 loads eyelashes and navel morphs too. But this also happens if we load it as custom morph so I’m not sure if this is a bug from the PA with the bef expressions themselves.

    steps:

    1. load G81F and merge rigs
    2. scan the morph database
    3. load FACS and make all bones posable
    4. load the bef01 and bef06 expressions

    note. I see that “scan database” is back to the morphs section, this way it is not available unless we load a figure first.

    note. As for “01“ .. “50“ I agree they’re poor ids so may be the PA doesn’t really know what he’s doing. Unless may be @Enge can explain something we miss.

  27. engetudouiti

    Ok , thanks. I do not put hurdle so high ^^; so basically I may report from simple case one by one.

    maybe I may make expression only use pose bone or only use alraedy improted basic morphs first > then next use vendor morph which not imported. and may try witch include my shape key morph value etc.

  28. engetudouiti

    I really sorry I can not test G8 or G8.1 so do not know detail about Hidden morphs ^^;

    (so it depend G8 and G8.1 user,, that enhance or remove bug . I tried my best before about Fact realtion thigns, but I am not real user so not report about G8 and G8.1 real user is best to report about it)

  29. Alessandro Padovani reporter

    @Enge, I’m not sure how complex your custom morphs are, or how you erc freeze them. But if they work fine in daz studio and don’t work in blender, then you may try to save them as support assets and see if this fixes the issue.

  30. engetudouiti

    Alessandro, no, it may not relate with save support assets things.. (because I must make my any morph work in daz studio first Then I save all morph first. it is my usual process. (not save as scene without save as morph )

    At same time I do not do compex thing to make expression morph. so at least if new morph system work, I suppose it work well. about this case I only need to confirm new morph can import most of my hand-made expresison along with vendor expression. ^^

  31. engetudouiti

    Then (so now I am up-date data base),,,

    from see pics, new morph system after all load all shape related morphs if it was not loaded before. right?

    so later when user import same morph, maybe generate same morph in different category I suppose.

  32. engetudouiti

    mmmmm,

    Traceback (most recent call last):
      File "C:\Users\TAKE\AppData\Roaming\Blender Foundation\Blender\3.2\scripts\addons\import_daz\error.py", line 207, in execute
        self.run(context)
      File "C:\Users\TAKE\AppData\Roaming\Blender Foundation\Blender\3.2\scripts\addons\import_daz\animation.py", line 1519, in run
        StandardAnimation.run(self, context)
      File "C:\Users\TAKE\AppData\Roaming\Blender Foundation\Blender\3.2\scripts\addons\import_daz\animation.py", line 1228, in run
        self.defins, self.defins2, self.formulas, self.formulas2 = loadScannedInfo(name)
      File "C:\Users\TAKE\AppData\Roaming\Blender Foundation\Blender\3.2\scripts\addons\import_daz\scan.py", line 296, in loadScannedInfo
        return defins, defins2, formulas, formulas2
    UnboundLocalError: local variable 'defins2' referenced before assignment
    

  33. engetudouiti

    Or I may better to report it as my topic for G3 specific about new import expression things?

    I think G8.1 and G3 can not work by same control pose system ..

  34. Thomas Larsson repo owner

    @engetudouiti Fixed in last commit. The plugin loads two tables because G8 and G8.1 are the same. Forgot to test with G3.

    @Alessandro The scan button is visible in the Morphs section even if nothing is selected.

  35. engetudouiti

    OK now I confirmed. Yes it seems work without I miss options.

    One thing I recently confuse things is we have 2 options for bone pose.

    Pose driven bone, and Pose bone.. I recently do not know clear difference.

    before drv bone was simple.. so maybe add on can pose it directly even though there was no contorller. but

    about recent version, when we pose facial bone, we may only use deform bone right?

    so when we apply bone pose, (it not include pose controller bone pose) value, it may use deform bone, not directly change drv bone value.. right?

    (that means,, Pose controller morph >> pose drv bone >> pose bone value >> pose deform bone.

    So I do not know which option (toggle) actualy I need when import any expression … (usually I toggle both though,, ) I means these 2 options

    btw about this test I only use already improted morph value + pose bone value, so I next try which change morph controller value which have not imported for blender.. (but not include new shape key delta)

    about this case, I suppose add on not generate new morph.. (but bake to bone pose or imported morph value)

  36. engetudouiti

    Do not you change mode as something special? for me even though I select light, now I can see it the button. (some up-date issue? though version is same as me)

    (ideally I may prefer it in pose morph section though.. because we may often use it to import pose not import morph^^; but it is OK,, (so when import new morph may need to use it I know)

    Do yours think good, to report here about G3 issue? now I need to add failed case
    (but basically improved I know, because it actually catch Symon expression contoller which I had not imported ) but not perfect ^^; (I mix use symon + my custom controller + basic morphs)

    blender

    I made it non symetry but when I import moprh, I think lip and mouse not pose as same as daz.

    (though I still not check all morph value, symon expression morph value shouuld be baked already I suppose..)

  37. engetudouiti

    some .r or .l value seems not used.. , I feel… (so I test with MHX, but hope it not related with ^^;

    maybe I later test with pure new generate MHX though..) maybe need to test each imported morph first though (if it failed,, import expressoin will show same problem)

  38. engetudouiti

    Ok now I seems find failed SWM product morph (about this case) which change lip shape like this.

    I may report later, with test which only use the morph value.

    (but do in my other topic, it need to add many pic and file I suppose…)

  39. Thomas Larsson repo owner

    engetudouiti, The Affect Driven Bones option was obsolete and is now removed. It determined if keys should be set for bones whose parent is a drv bose, i.e. a bone that had been made posable.

    Alessandro, the button was there if the active object is something different from an armature or a mesh (including nothing at all), but you managed to catch the only case when it wasn’t: when the active object is an ancient daz armature with obsolete morphs. Now this case is handled as well.

  40. engetudouiti

    @Thomas

    thanks it really make me feel easy (and I remember maybe,, I ask it before when drv bone was parent of deform bone^^; but anyway, so I never worry about the option)

  41. Alessandro Padovani reporter

    @Thomas, with commit 8768dff I get the scan button so now it works fine.

    My example above was commit c83862d with the default blender new scene with the cube, so nothing fancy. My notes above for commit 7daa08b still apply to 8768dff, so I’m still confused if I miss something.

  42. engetudouiti

    Alessandro temporally wait to close this please 🙏

    I need to confirm one thiing (I do not think at current it work for my case) Thomas,

    My expression file only move One Simon controller. then I do not import it to blender.

    The simon controller include many bone transform. and some basic morph values.

    Then expression file (preset) which I load as expression not include any pose bone transform,

    but used Controller include many facial bone transform, Do you think it actually apply those bone pose?

    I do not think it use those pose bone value.

    1. this is my preset file (which I import to blender as Import expression with use data base)
    {
        "file_version" : "0.6.0.0",
        "asset_info" : {
            "id" : "/People/Genesis%203%20Female/Expressions/TK/lipbiter.duf",
            "type" : "preset_pose",
            "contributor" : {
                "author" : "TK",
                "email" : "",
                "website" : ""
            },
            "revision" : "1.0",
            "modified" : "2022-08-17T14:36:25Z"
        },
        "scene" : {
            "animations" : [
                {
                    "url" : "name://@selection/head#eCTRLSWMBiteLipRight:?value/value",
                    "keys" : [ [ 0, 1 ] ]
                }
            ]
        }
    }
    

    As you can see it do only one thing, set the eCTRLSWMBiteLipRight controller as 1.00

    Then I do not import the controller as morph. (because your new system serch and apply value for basic import morph and pose bone value.

    the ctrl file is like this (I cut about morph value setting, because now it seems correctly applied , so only bone pose value is matter

    {
                        "output" : "lLipLowerOuter:/data/DAZ%203D/Genesis%203/Female/Genesis3Female.dsf#lLipLowerOuter?translation/x",
                        "operations" : [
                            { "op" : "push", "url" : "head:#eCTRLSWMBiteLipRight?value" },
                            { "op" : "push", "val" : 0.0161723 },
                            { "op" : "mult" }
                        ]
                    },
                    {
                        "output" : "lLipLowerOuter:/data/DAZ%203D/Genesis%203/Female/Genesis3Female.dsf#lLipLowerOuter?translation/y",
                        "operations" : [
                            { "op" : "push", "url" : "head:#eCTRLSWMBiteLipRight?value" },
                            { "op" : "push", "val" : -0.2445026 },
                            { "op" : "mult" }
                        ]
                    },
                    {
                        "output" : "lLipLowerOuter:/data/DAZ%203D/Genesis%203/Female/Genesis3Female.dsf#lLipLowerOuter?translation/z",
                        "operations" : [
                            { "op" : "push", "url" : "head:#eCTRLSWMBiteLipRight?value" },
                            { "op" : "push", "val" : 0.5536269 },
                            { "op" : "mult" }
                        ]
                    },
                    {
                        "output" : "lLipLowerInner:/data/DAZ%203D/Genesis%203/Female/Genesis3Female.dsf#lLipLowerInner?translation/x",
                        "operations" : [
                            { "op" : "push", "url" : "head:#eCTRLSWMBiteLipRight?value" },
                            { "op" : "push", "val" : 0.01400518 },
                            { "op" : "mult" }
                        ]
                    },
                    {
                        "output" : "lLipLowerInner:/data/DAZ%203D/Genesis%203/Female/Genesis3Female.dsf#lLipLowerInner?translation/y",
                        "operations" : [
                            { "op" : "push", "url" : "head:#eCTRLSWMBiteLipRight?value" },
                            { "op" : "push", "val" : -0.3438323 },
                            { "op" : "mult" }
                        ]
                    },
                    {
                        "output" : "lLipLowerInner:/data/DAZ%203D/Genesis%203/Female/Genesis3Female.dsf#lLipLowerInner?translation/z",
                        "operations" : [
                            { "op" : "push", "url" : "head:#eCTRLSWMBiteLipRight?value" },
                            { "op" : "push", "val" : -0.08691518 },
                            { "op" : "mult" }
                        ]
                    },
                    {
                        "output" : "LipLowerMiddle:/data/DAZ%203D/Genesis%203/Female/Genesis3Female.dsf#LipLowerMiddle?translation/y",
                        "operations" : [
                            { "op" : "push", "url" : "head:#eCTRLSWMBiteLipRight?value" },
                            { "op" : "push", "val" : 0.005443871 },
                            { "op" : "mult" }
                        ]
                    },
                    {
                        "output" : "LipLowerMiddle:/data/DAZ%203D/Genesis%203/Female/Genesis3Female.dsf#LipLowerMiddle?translation/z",
                        "operations" : [
                            { "op" : "push", "url" : "head:#eCTRLSWMBiteLipRight?value" },
                            { "op" : "push", "val" : 0.2462883 },
                            { "op" : "mult" }
                        ]
                    },
                    {
                        "output" : "rLipLowerInner:/data/DAZ%203D/Genesis%203/Female/Genesis3Female.dsf#rLipLowerInner?translation/y",
                        "operations" : [
                            { "op" : "push", "url" : "head:#eCTRLSWMBiteLipRight?value" },
                            { "op" : "push", "val" : 0.2071017 },
                            { "op" : "mult" }
                        ]
                    },
                    {
                        "output" : "rLipLowerInner:/data/DAZ%203D/Genesis%203/Female/Genesis3Female.dsf#rLipLowerInner?translation/z",
                        "operations" : [
                            { "op" : "push", "url" : "head:#eCTRLSWMBiteLipRight?value" },
                            { "op" : "push", "val" : -0.109355 },
                            { "op" : "mult" }
                        ]
                    },
                    {
                        "output" : "rLipLowerOuter:/data/DAZ%203D/Genesis%203/Female/Genesis3Female.dsf#rLipLowerOuter?translation/x",
                        "operations" : [
                            { "op" : "push", "url" : "head:#eCTRLSWMBiteLipRight?value" },
                            { "op" : "push", "val" : -0.009935133 },
                            { "op" : "mult" }
                        ]
                    },
                    {
                        "output" : "rLipLowerOuter:/data/DAZ%203D/Genesis%203/Female/Genesis3Female.dsf#rLipLowerOuter?translation/y",
                        "operations" : [
                            { "op" : "push", "url" : "head:#eCTRLSWMBiteLipRight?value" },
                            { "op" : "push", "val" : 0.4313995 },
                            { "op" : "mult" }
                        ]
                    },
                    {
                        "output" : "rLipLowerOuter:/data/DAZ%203D/Genesis%203/Female/Genesis3Female.dsf#rLipLowerOuter?translation/z",
                        "operations" : [
                            { "op" : "push", "url" : "head:#eCTRLSWMBiteLipRight?value" },
                            { "op" : "push", "val" : 0.06805158 },
                            { "op" : "mult" }
                        ]
                    },
                    {
                        "output" : "LipBelow:/data/DAZ%203D/Genesis%203/Female/Genesis3Female.dsf#LipBelow?translation/y",
                        "operations" : [
                            { "op" : "push", "url" : "head:#eCTRLSWMBiteLipRight?value" },
                            { "op" : "push", "val" : 0.5095555 },
                            { "op" : "mult" }
                        ]
                    }
    

    You may see many bone actually transformed in this ctrl file.

    why I think your add on not apply it,, when I import expression file, (which I describe above), I do not think, these bone pose value are actually used. so I click clear morph = it may only clear morph value not clear bone pose >>> now my character return no facial pose.

    to confirm I click Clear pose >> it may only reset bone pose >> no change.

    So I suppose at least when import expression,, add on find used controller, but if controller describe these pose bone drive value, it seems ignored.

    it only work when I already imported these controller (which move facial bone as pose)

    I check json (data base), then I understand how it will work,, but No controller describe pose bone transform in json.

    (but actually many controller dsf will add bone pose value in each controller file along with other morph value)

    so if add on only use the json to set values. maybe pose bone value are simply ignored. (only used when the controller had been imported, because those pose bone value will be auto applied,

    Or I think if my expression file include bone pose value, it may be applied later. but if non imported controller describe pose bone value, all bone pose simply ignored I think.

    You may check many pose controller (morph) dsf in data, and may notice, actually many pose bone value were described in Vendor pose controller file. (aka custom morph / custom pose controller)

    (though some file may not use bone transform, but some vendor may use it often as same as use other basic morph value to make pose controller file(dsf)

  43. engetudouiti

    Then if you need to check real case, I may make custom controller dsf which include only one face unit value + many bone transform. aka “pCtrTKcustom.dsf”

    and make expression which only set the pCtrlTKcustom as 1.00 as TKexpression.duf in Expression directory.

    Then import basic morphs (include face unit), scan data base >> import TKexpression.duf by import expression. I suppose all bone pose ignored. only face unit applied (then face unit may pose some bone only)

  44. engetudouiti

    I now test it and confirm add on not use pose bone transform in my custom pose cotnoller but only add face unit value which set in same controller file.

    In daz studio my expresssion preset load

    the expression only include one custom controller value. (set as 1.0)

    then the controller only use face unit >> Eye close + some facial bone pose

    I already improt all face unit (include eye close) but not import my new controller but it is already saved as morph (so it locate in data g3 morph directory correctly)

    I scan data base,, then it describe my controller too.

    now I import the saved expression file >> it only apply eye close value, but not add any bone pose

    , though I think most of G8.1 head pose controller may not need to use facial bone pose but only use G8.1 FACT morphs.. so it may not matter, , but if you consider vendor pose controller which include facial bone pose (translate, rotate, scale), then when vendor make expresison with use the controller, hope you will enhace and make it work..

    as I said,, expression file can include bone pose,, at same time vendor expression controller (aka morph) can include bone trnsform too . it is same many face unit actually pose facial bone.. so at least most of expresison preset file which I mix use simon controller, can not load it, untill I import all simon expression controller ^^; (though I do not think it is bug, but current limitation)

  45. Thomas Larsson repo owner

    Alessandro, there is a BEF01 shapekey that wasn’t loaded before. It is subtle but nonetheless it is there.

  46. Thomas Larsson repo owner

    Engetu, I don’t know about your last case, but the lip bite pose is imported correctly in Blender. You can see in the terminal window how the plugin finds the morphs and builds drivers for the bones.

    * lipbiter.duf 1
    
    Loading morphs in category Loaded
    Making morphs
     * eCTRLSWMBiteLipRight
    Making missing morphs level 0
    Building drivers
    Building sum drivers
     + lLipLowerOuter
     + lLipLowerInner
     + LipLowerMiddle
     + rLipLowerInner
     + rLipLowerOuter
     + LipBelow
    No rest drivers
    Folder  loaded in 0.695 seconds
    File D:\DAZ 3D\Studio\My Library\People\Genesis 3 Female\Expressions\TK\lipbiter.duf imported in 0.743 seconds
    

    The morph doesn’t show up in DS though, so presumably I didn’t construct the duf file right. In the future, could you post the entire file, or better still as attachment, so I don’t have to reconstruct it.

  47. engetudouiti

    You may see my close topic too about Simon expressions, I test many already.

    https://bitbucket.org/Diffeomorphic/import_daz/issues/1112/import-expression-as-morph-issue-new-morph

    I close it because the first problem was already solved with up-date

    then test with Simon expression product (it not contain expression file but slider only, so to use it I need to save as expression. (to I test your new system )

    There was vendor side problem. (though it work for ds) . only about lip bite cotroller, there were vendor miss, because He set ID and Name as opositte. r and l. then it cause issue about morph values.

    but I already correct it, then finally confirm, your import expression failed, when I load Expression which use Controller, then controller describe pose bone values with b asic morphs values.

    my attached 2 files are, pure my hand made files, so you can free test and confirm ,what is problem please.

  48. engetudouiti

    Of course if I import all simon exp slider as morph first , after that I can import any expression preset which set value for them. but I think it is not you intended.

    you intend, even though I do not import simon exp sliders morph, I can import as expression which set value for simon expression sliders. (then they drive basic morphs/ pose bones)

    As real case, we mix use more controller and save as Expression though. (so this is not complex case)

    ===
    To test attached controller (dsf) , you may need to correctly cloate it in this directory.

    data\Daz 3D\Genesis 3\Female\Morphs\TK\Test

    (make TK\Test directroy temporally please ^^;)

    Then load G3 and confirm how the pose controller work first. and return zero pose, then apply my attached expression, you should see, it just set the controller value as 1.0. (so my expression and controller have no issue)

    btw I did not make Alias, so you need to select Root node, and in Pose controller/head/expression/TK

    you may find my test controller.. it work like this

    after that in blender, load G3, import basic morphs (face unit etc), then , Scan DAZ database (for G3) ,

    import my attached Expression duf , by import as Expression with use data base/morph/bone please. you may see only morph (eye close) worked.

  49. engetudouiti

    I suppose why when you import your duf, it start to build sum drivers, because we often try to save all pose bone value too.. (or activate only modified value >> it check final value change and describe too)

    But in duf, those bone value are simply keep as zero transform. because pose controller already used..

    Then when import the expresison, it may find those pose bone name, then start to drive pose bone. (but the value are zero), so after all only morph value will be driven. (though I test again with most recent version)

  50. engetudouiti

    oooooo recent version now worked for me,, why? (though I test with default rig this time) ^^;

    now I test with MHX rig, it not work.. so it seems MHX converted bone name issue ?? your import expression work for mirror facial name ?

    with def rig >>

    No I just change view so miss understand as if cheak change.. actually it not pose bone. only close eyes I may wait Thomas confirm and reproduce this issue.

    I already confirm, when Import the pose controller as morph, it work well.

    after that, if I import same expression it work. = Old way which not scan data base

    but if I do not import the pose controller as morph, (just saved in daz data >> and Scan data base)

    then load the expression, it not apply bone pose at all. = New way you now improve

  51. engetudouiti

    Then I do not clear remember, but if Import morph (pr pose cotnroller) dsf, have option which not generate slider, but just set value with use already generated morph data base, >> set sub-component morph and set bone pose,,,, at least we can import “Simon Pose controller” as pose.. though it only work for expression which just use one controller ^^;

    But if vendor only offer pose controller (expresison), and product is large, then user only hope to use the pose (slider value as 1.00) without generate new custom controller, such option is useful..

    (though at current I simply hope you enhance import expresison (preset), which can manage al posel preset duf with data base..

  52. Alessandro Padovani reporter

    @Thomas

    Ok so we’re unlucky because all the bef expressions seem to use custom shapes other than facs. This way the database is unuseful because there’s no way to skip the intermediate morphs, we can load the bef expressions as custom morphs and get the same result.

    One last thing please, did you check why bef01 loads the navel and eyelashes morphs as well ? It doesn’t happen with the other bef expressions, it’s only bef01. Is this a bug from the PA ?

  53. Thomas Larsson repo owner

    Engetu. Problem fixed in last commit. Since the fix is a change in the scanned database, you must rescan.

    The plugin replaces a morph that is a combination of other morphs with its parts, and in that case the morph does not have to be loaded. However, that doesn’t work for a morph that consists of other morphs plus some poses, because the poses are lost. The lip bite morph worked for me because it was only poses, so it was loaded. The TKexp morph is a combation of the Eyes Closed morph plus poses, and must not be replaced by Eyes Closed alone.

    ERC morphs don’t count as poses but are ignored. This is not quite right if ERC morphs are enabled, but I don’t want to do that because many morphs using FACS would need to be loaded, e.g. FEM.

  54. Thomas Larsson repo owner

    The BEF01 pose preset loads the 01 morph

                    "url" : "name://@selection#01:?value/value",
                    "keys" : [ [ 0, 1 ] ]
    

    The 01 morph loads the FACS morphs but also the BEF01 morph

                        "output" : "Genesis8_1Female:/data/Daz%203D/Genesis%208/Female%208_1/Morphs/Neikdian/Big%20Expressive%208.1/BEF01.dsf#BEF01?value",
                        "operations" : [
                            { "op" : "push", "url" : "Genesis8_1Female:#01?value" },
                            { "op" : "push", "val" : 1 },
                            { "op" : "mult" }
                        ]
    

    It also sets the PBMNavel morph to -1, so it must also be loaded. It would not be loaded if the value were 0. In practice a negative value is the same as zero, but the plugin does not know that.

                        "output" : "Genesis8_1Female:/data/DAZ%203D/Genesis%208/Female/Morphs/DAZ%203D/Base/PBMNavel.dsf#PBMNavel?value",
                        "operations" : [
                            { "op" : "push", "url" : "Genesis8_1Female:#01?value" },
                            { "op" : "push", "val" : -1 },
                            { "op" : "mult" }
                        ]
    

    Finally, the BEF01 morph defines the shapekey.

    So using the database skips one step. Instead of loading

    BEF01 pose preset > 01 morph > FACS and BEF01 morphs
    

    we just have to load

    BEF01 pose preset > FACS and BEF01 morphs
    

  55. Alessandro Padovani reporter

    Thank you @Thomas for the complete explanation, so bef01 is a bug from the PA that the user can eventually fix by editing the dsf file and removing the navel and eyelashes. Please @Enge let me know when you’re ok so I’ll close this one as resolved.

  56. engetudouiti

    @Thomas

    Thanks, from seeing reply, it is waht I mentioned. So I suppose now it should work for most of case.

    The real product “lip bite morph” consist of Mix pose and base morph value.

    most of Simon expression slider (and aniblock) product are made so..(I am sorrry I can not full up-load the file then I only pick up the place which not worked (I cut)) in clode,, then expect if you confirm to see it)

    So now I confirm it work at same time so about this product, I suppose if user use one of controller for expression, and import expresession, the morph must need to be loaded ^^; I feel about this case it is same as old way (Import controller first or not) but maybe it is helpful I can not remember which product controller and which expression file include pose or only morph or pose and morph value.. so maybe it need to be loaded about those case..

    Maybe you need to describe it because user do not know morph will be loaded or not untill try.

    eg G3 zexpression cotroller usually include some bone pose + basic morphs. I happend to install them already, but I suppose if I have not,, then make expression, add on will load them when it was used in new category (loaded default)

    @Alessandro

    Ok I think now it work for most of case withohut problem.

  57. engetudouiti

    Yes I now check all G3 Expression controller (vendor) product which I bought already

    then actually most of product include pose bone value (lip around or eye etc) to adjust detail.

    So if user have not imported those controller, but load Expression which use some of them, they will be loaded as new category… If user often try to mix slider and make preset,, then load as Expression, it may generate many new controller without categorize Vendor or Product name.

    If you can enhance future (not soon), you may try to separate Morph build and Pose bone build, and describe in json… (though it make json huge), then count pose bone later, = we may not import vendor morphs when user just load expression which use them.. I suppose. (but maybe G8.1 user may need not it, so I suppose there is not many user who request it you ^^; )

  58. Thomas Larsson repo owner

    Engetu, I don’t think it is easy to separate poses and submorphs. If a morph contains poses it must be loaded, and then you get the submorphs automatically. To change that would be a major development.

    What could perhaps be done rather easily is to add a new tool that bakes poses and shapekey values from rig properties. That could be useful for exporting to game engines, I think.

  59. engetudouiti

    Yes I see it is easy to imagine^^; Though technically I suppose we can separate driven morph value and direct pose bone value described in controller, and re-build as group….

    And if bake pose and shape key tool should be useful too… when we adjust shape or pose slider is really powerful, but if new tool can keep fin pose and shape,, with release from driver ,, it is new powerful tool I feel.. (usually I hope all preset describe only pose and shape key value, without any dirver or ERC if we can)

    I may happy test when it will come thanks. but I suppose bake all driven value as pure deform bone pose value is good start point because it not need to think about shape key.

    I think we do not make current morph system to easy add off set (user input value + driven value) about shape key… (it need to show all shape key controller which driven by other morph ,,)

    I think it is kind of blender limitation.,. (driven value + user input off-set value)

    so if you bake driven shape >> you may plan to remove driver for the shape key,, I suppose ^^; it is not I hope.. (I may hope to see,, bake driven shape key value for each pair controller value for shape key (user input with pair controller value)

  60. Thomas Larsson repo owner

    The scanned database now stores the min and max values, and use those to limit the morph values. In this way the value of PBMNavel becomes zero and that morph is not loaded anymore.

  61. Alessandro Padovani reporter

    Commit cf6d0a6 works as intended.

    The eyelashes are still imported though and set to -1, so they’re zero too. Also both the navel and eyelashes are imported anyway if we load the bef01 custom morph. So I’m not sure the extra time to import the min max is worth it, the correct fix in this case is to fix the dsf file.

    note. As a side note the scan time for G8F G81F in my system went from about 1 minute in the first version to about 4 minutes in the last version. So stills fast enough.

  62. Log in to comment