Test for new prop atatcched for rig.data and report etc

[engetudouiti]

I check code where you modified in commit 3ec642f, and you first use function to change rig to rig.data then when set this option, it change self.rig (armature object) as self.rig.data

 def getArmature(self, rig):
+    if GS.useArmatureDrivers:
+        return rig.data
+    else:
+        return rig

the problem seems, there are some conflicts place, when you change rig to rig.data

you mix use self.amt, and self.rig , then with condition self.amt set as None.

 File "C:\Users\TAKE\AppData\Roaming\Blender Foundation\Blender\2.92\scripts\addons\import_daz\driver.py", line 212, in addTransformVar
    pb = rig.pose.bones[bname]
AttributeError: 'NoneType' object has no attribute 'pose'

When this function is called,, rig set is none without your intention… (your intention is,, , rig should be object.data (armature.data)

I may recommend, not modify many code for default option as possible as you can.., when we set useArmatureDrivers as False.

your commit modify many place, for default, to work for both option, at same time but it seems some miss usage for arg self.amt and self.rig, without your intention.

I afraid this commit actually effect for user who not need to test it… (because at current it can not import morphs,, for those conflict eroer..) If you do not bother, I may test it separately.. so I recommend, you once remove this commit. (though I hope to test, what may change, when set prop for rig.data)

[engetudouiti]

I could solve first issue.

I change load_morph.py 639

    def makeBoneDriver(self, string, vars, rna, channel, idx):
        from .driver import addTransformVar
        rna.driver_remove(channel, idx)
        fcu = rna.driver_add(channel, idx)
        fcu.driver.type = 'SCRIPTED'
        string = self.multiplyMults(fcu, string)
        fcu.driver.expression = string
        ttypes = ["ROT_X", "ROT_Y", "ROT_Z"]
        for j,vname,bname in vars:
            addTransformVar(fcu, vname, ttypes[j], self.rig, bname)
            # I changed self.amt to self.rig
        return fcu

addTransformVar(fcu, vname, ttypes[j], self.amt, bname) argument

self.amt to self.rig then it not show erroer at least when I load duf. I still not try to import morph, so I need to check later. (it may remove easy mode bug too,, at least without import morph)

[Thomas Larsson]

Yes, I noticed that the argument to addTransformVar was wrong too. Thank you.

[engetudouiti]

Thanks I could not solve JCM issue at all ^^; actually I did nothing .

Do you think, if you make all morph props attach with Armature.data it cause issue? Like PHM shape key props. and pose controller prop.

I think at least about pose controlelr prop, attach for armature data may work better (just guess)

then if prop which drive shape key, and you can generate UI slider which change prop(raw) attached with armature.data, even though in object mode,, I just guess, if all prop can be attached as armature.data.

Though I do not hope break again, but when you load morphs and generate prop, if it work,, it make things easy I thought.

I did not test as UI prop, but if UI prop slider can change armature.data prop even though in object mode, and it work,, I feel it may not cause problem. (but can not confirm it at all, if it cause new issue, I afraid.)

I just worry, if it is difficult, when you load many mroph.dsf, only pose bone drive props attach to armature,data. if it is not problem at all, you need not.

‌

‌

[engetudouiti]

I feel, at current pose controller (Facs Unit) can not drive bone, . (with armature.data.prop)

When I check drv eyelids bone rotation, it have no driver , then it actually cause issue, when I use Facs Unit. eg eye blink etc. maybe most of Facs Unit which drive bone, show same issue. (I test with custom eval function option now)

I confirmed, make all bones posable, after import morphs … (some easy mode setting may confilict with?)

Traceback (most recent call last):
File "<bpy driver>", line 1, in <module>
File "C:\Users\TAKE\AppData\Roaming\Blender Foundation\Blender\2.92\scripts\addons\import_daz\propgroups.py", line 163, in evalMorphsLoc1
return sum([pg.eval(rig) for pg in pb.DazLocProps1])
File "C:\Users\TAKE\AppData\Roaming\Blender Foundation\Blender\2.92\scripts\addons\import_daz\propgroups.py", line 163, in <listcomp>
return sum([pg.eval(rig) for pg in pb.DazLocProps1])
File "C:\Users\TAKE\AppData\Roaming\Blender Foundation\Blender\2.92\scripts\addons\import_daz\propgroups.py", line 95, in eval
value = rig[self.name] - self.default
KeyError: 'bpy_struct[key]: key "facs_jnt_JawOpen(fin)" not found'

‌

It seems still conflict, rig and rig.data I suppose ^^; at current (fin) attached for rig.data, then (raw) attached for rig (object) I suppose. about this case,, evel function need to get rig as object. maybe it failed?

(at current, you may recommend to use evel morph (custom function) right? I test with the option.)

[engetudouiti]

I may recommend, if you add prop for armature.data, Test raw prop attach with armature.data make things more easy

(though it need test, to confrim actually it not cause issue) it may ease some conflict proplem rig and rig.data, with gloval option I suppose….

because you have used rig as object for most of case. then now you need to exchange rig or rig.data case by case…with amt variable..

[engetudouiti]

At current we have issue, when pose bone drive object[“prop_fin”], so as default, you set driver for shape_key. it work.

Then as option, now test, pose bone drive armature (armature.data)[“prop_fin”] then set shape key driver as [“prop_fin”]

if it can solve issue,, it means,, armature[“prop”] can drive shape key without problem. for MHX or default rig too.

Then all raw prop may drive other props, or pose bone. so I think set raw prop for Armature(.data) as same as fin prop. seems not cause issue. (I confrimed, even thoguh I keep object mode, I can still change Armature.data[“prop”] without problem.

I have thought, when prop drive attached mesh shape key, it seems good add prop for Armature object. but we already know, same prop drive shape key with pose bone too.

To generate daz morph prop from dsf, and set driver expression, I may suppose,, set prop for same data may make things easy. (even though it show error, you can clear find reason.. because now One option may only add prop for ArmatureObject, then test option may only add prop for Armature.data.

With my simple test,, (simple rig mesh with shape key) Then I atach all prop as Armature.data[“prop”]. then test each cases.. it seems work for me,, (at least there is no clear difference,, when we attach raw prop for object)

Though it may need to test with daz figure, (they have many props and driver hieralchy), when prop drive armature pose bone, I thiink, set prop for Armature.data seems more reasnable.. so if armature prop can drive shape key, as same as Object prop..

Then if we test to atach some JCM or shape key driver prop {fin} for armature.data , I think there is not strong reason, we need to use Rig object to attach raw props. (after all, they are controlled by UI prop, which point to the csutom prop) so it not matter for user side.. (we can still controll in object mode, and pose mode with select rig)

I only afraid ,if it cause new performance issue though,, (I can not imagine it cause new issue,,, for most of controller, which may drive bone rotation,, )

[engetudouiti]

Anyway with use eval morph option, I could solve 2nd issue (not drive pose bone)

‌

class DazMorphGroup(bpy.types.PropertyGroup, DazMorphGroupProps):
    def __repr__(self):
        return "<MorphGroup %d %s %f %f>" % (self.index, self.prop, self.factor, self.default)

    def eval(self, rig):
        if self.simple:
            return self.factor*(rig.data[self.name] - self.default)
        else:
            value = rig.data[self.name] - self.default
            return (self.factor*(value > 0) + self.factor2*(value < 0))*va

in propgroups.py, DazMorphGroup class def eval,, you need to change

if and else. condition return satements rig to rig.data.

self.factor*(rig[self.name]-self.default) to self.factor(rig.data[self.name]-self.default)

and value = rig[self.name] - self.default to rig.data[self.name] -self.default

Now pose bone driven by UI controller again. (test with Facs Unit now I do not catch erroer)

But I do not know if it only work with use new prop option (attach prop for armature.data.

[engetudouiti]

Then one new problem , I can only eye side to side for G8.1 face unit, for plus direction. (so only look to right but not look to left for both eye.

I can not find what is going on with eval morph option. (driver expression not offer me usable infomation..)

I check eye bone min and max but it seems not problem. eye look Left /Right Fin prop value actually get -1 value. but drv bone X rotate not driven to the - direction. only move + direction, it is new issue for me..

‌

[Thomas Larsson]

Something happened to the custom drivers; things work if you use Xin’s sum drivers, i.e. turn off Custom Drivers in the global settings. Even if the custom drivers are deprecated, it is not time to retire them yet, so I will check what happened.

[engetudouiti]

I see. I actually did not know, which option I may test first ^^; now there are many combination and easy mode options + global setting. anyway in here I only report about , props with armature.data options. Thanks!!

[Thomas Larsson]

Now there is at least one option less: the Armature Drivers option is determined by Custom Drivers. Old custom drivers use object properties for both raw and fin, whereas sum drivers use raw object and fin armature properties.

[engetudouiti]

I see, do we still have c68c9d5 problem with sum drivers?

[engetudouiti]

Thomas one thing, I may recommend is (though if it do not show any difference or not)

at current you seems not change driver prop ID type as armature, but still keep prop ID type as Object, then set path data[“morphFin”].

but blender offer Id prop type as Armature. so if you can get Armature.data,name, easy, you can make it like this. (actually I have used it to set prop for rig.data)

Do you see issue? the reason why I ask it,, I supose if It work better. (though I think we do almost same thing, by set Path as data[“morphFin”].

‌

[engetudouiti]

And I think there still issue, when you set expression with Sum option (and armature.prop)

eg, Brow down (Fin) drive Brow down R and L.. But at current it only drive Brow down Left.

Actually you seems not include ["facs_ctrl_BrowDown(fin)"] in driver expression for Brow down R.

(set another prop, like this,,)

‌

‌

To test this I do not use easy mode,, but import each morph group one by one.

I suppose when I import Facs expression, then it seems change expression

Variable B from [facs ctrl Brow dow]n to["facs_ctrl_Frown(fin)"].

add on needed to change (a + b) to (a + b + 0.1*c) ,without change variable for a + b, but set new variable C as Flown(fin) and add it to (a + b)

but when import Facs_ctrl_Frown, it seems not add new variable but exchange b. then it happend. It is related how add on generate (overwrite) expresison and set variable,, when we add new dsf which controll already imported morphs. I suppose.

And I guess this is most difficult part, how add on generate new expression, when we import new morph , without break already imported morphs expression but only add or multiple new variable.

[Xin]

Thomas, have you looked into scanning the entire daz directory to get all dependencies and save the result to a cached file, instead of scanning dependencies on demand each time the user loads a morph?

Later, when the user loads morphs, you look into the cached file and see quickly what the dependencies are, and you load them too along the requested morph, so nothing ends up half loaded.

Don’t know if this would be useful or if it would solve these issues, just something to keep in mind in case it does.

[engetudouiti]

btw This problem may only happen, when user not use easy mode >> use Unclassified standard morphs to gather all morphs . but import each category individualy.

And this problem happen, with custom driver too. I confirmed, with custom driver option, and import Facs unit, and import Expression, the expression change (overwrite variable. which need to keep as same as before)

So it is not directly related, use Object or Armature to attach prop. Anyway if user import Facs Unit first, then later try to import Expression, this problem happen .

One reason why it is really difficult,, at current there seems no clear format, to represent expression, which can divide, (raw), (sum-part only) (multiple part only) .

if script can check each section, from current expression, maybe it only need to add new value and insert it for sum , or multiple. stage.

eg now one morphA expression is A + B. then when we import new morphB, it drive morphA. then it describe delta add

about this case, it may only need to add new variable C in this expression.

But if morph A expresison was (A + B ) *C,,, then import new morph B it drive morph A as delta add.

we need to generate new variable D, and convert (A + B ) *C to (A + B + D) *C .

I can do this easy, if I actually read duf formula, and change expression for each morph driver, but script need it auto-mate.

I think we need clear format,, which can check,, raw part, 1st stage part, and 2nd stage part as expression.. and everytime add on keep to use same format, to represent all driver (not include key type) then insert new Variable correctly.

And one way is (I just guess), use specific variable for each purpose. I suppose maybe anyway “a” represent raw value , if you do not generate raw prop , you do not use “a” .like JCM .. if there is “a” in driver expression , it should means the fin prop raw value or raw = 0. then only use ma, mb, mc, or any type, for multiple stage variable.

at current I think add on try to generate al ltype variable as aphabet,, so even though scirpt can check current expresison, to add new variable with use current expresison is really difficult… (script can not find where is sum part or where is multiple part, I suppose),, and maybe we need to use “(“ “)” to group each stage, in expression more often.

now we use “A + B” (it is simple and clean) but it seems better even though there is no multiple stage, use “(“ + “A +B” + “)” = expression (string). so when new morph multiple it,, add “ma” as multipler variable, then expression(string) + “*” + “ma “ = new expression work.

[Thomas Larsson]

Armatures are now used as target ids. Unfortunately, this brought back problems with dependency loops for mhx.

[Thomas Larsson]

Xin, the plugin scans all locations with standard morphs when the first standard morph is loaded, and stores the filenames in a cached directory. However, you seem to suggest that the plugin would open and parse all files to find dependencies when it starts. AFAIU, DS does something like that itself. If your DAZ database is large, this would take a long time, even in DS which is written in a compiled language, and people complain about that.

So no, I don’t want to open files before they are needed. This is only a problem if you only load part of the morphs of a given type though.

[Thomas Larsson]

engetudouiti, I am aware that facs expressions break facs units. Problably you could have the same issue with custom morphs that drive other morphs instead of bones or shapekeys directly. I will try to fix the expressions so it doesn’t happen, but in the long run it is not the right solution, because eventually we risk to exceed the 512 character limit. It does not happen currently because there are only six facs expressions (at least on my system), but sooner or later there will be more.

The right solution is to introduce intermediate properties and sum drivers for driven properties, just as we did for driven bones. That will remove the limits on the number of driver variables, but it will also complicate the code in the normal case that only a few properties are driving.

An easy fix in the short run would be to disable facs expressions. The facs units are important because they are needed to load face animations, but the expressions not so much.

[engetudouiti]

OK Thomas thanks, how manage new import morph groups (one package with user selected ) and modify expression of driver, is difficult problem. (I believe we can solve it as future though) but it seems better to talk only about it .

Then do you think, if you make all props (include raw prop) as armature prop will cause new issue? because at current I suppose we mix use Armature prop and Object prop.

Actually I do not know which work better, clear,, (I happend to see strange issue, about Object prop which driven by pose bone transform, so I just said guess, if you can use armature prop ) but if armature props may work most of case,, I think we may better to use one driver target type,, about all props which use driver or driver targets.

(so raw should be included in fin prop driver expression, I suppose,, use one type (object or armature) may make easy to manage them.

You may test only about the new option (armature prop option), so if it cause new issue, it may not cause many trouble I think.

‌

[engetudouiti]

About Xin suggested way, I think as future we may need to dump kind of data file, which describe, currently imported dsf file. path, and formula for each imported fin prop. (or save as blend file, which only include kind of group property of one armature or obj, empty, for each fin props. the group props have dsf path of the fin prop, and current expresison as str data.

The main problem is we have no data-base, to manage morph dsf file and generate driver expression, so I suppose, we may need way to store driver expression which user already imported. then when new import morph, use the saved data file, to re-build driver expression with currently imported dsf formula.

[engetudouiti]

I see why new intermediate prop may work better, we add props for armature..(not object).

because all intermediate props need to attach for each drv pose bones.

at same time, if we use such intermediate prop to divide expresison more short for prop (facs morphs etc)

we need to consider, how attach those props. To be frankly said, I can not imagine, it.. because bone already seprated for each bones.

but controller which may driver other fin props, are not separated..

let think, now we have one morphA

then morphA driven by delta add (need multiple with step value)

morphA = raw + (B.0.3 + C*0.4 + D*0.5 + E*0.6 + F*0.7…)

Xin way use “Sum Values” , as expression, then divide

it as Sum values = raw + B' + C' + E' + F'…. we need

B' C' D' E' …… as new intermediate prop for one morphA (Fin) (face units etc) but if It work , I may recommend,

at first generate expression as raw + (B.0.3 + C*0.4 + D*0.5 + E*0.6 + F*0.7), then check it over limit, only when it over-flow limit, add on try to divide by same count.

eg raw + ( B.0.3 + C*0.4 + D*0.5) + (E*0.6 + F*0.7) = raw + M1 + M2

sum values (raw, M1 , M2)

so it can decrease intermediate prop . for each morphA we only need to generate [morphA:M1] and [morphA:M2]

if it cause some new issue, after all I do not think new inter props can use to divide morph expression.

how convert default expression (not count over-limit) to new one which add new middle prop, may need to consider , but from each Fin value expression, str length, we can set count ,to divide sum. sometimes we need 4 middle prop, sometimes we need 2 middle props etc.

and we can enhance it later to work for 2nd stage expression.

So anyway first generate full expression (without consider limit), with clear format. it can easy check sum part and multiple part may need.

Then it may change by custom function or add middle props and “Sum Values” expression.

then if add on will try to generate inttermideate props ,

when generate MorphA raw and fin, generate, 2 new props first.

MorphA[raw] morphA[sum], morphA[mlt]

then every morph[fin] dirver expression can use same expression.

morphA[fin] = (morphA[raw] + morphA[sum]) * morphA(mlt)

then we may use “Sum values” only about “morphA[sum]” driver Expresison.

Though it is helrachical 2 times, so I do not know if it may cause problem…

I may test lator such hieralchical props and driver, with actuall case, ..

[engetudouiti]

Thomas do you already know C.object.update_tag() may need to update custom prop (ID prop), when we change prop value, and it may drive other prop etc… ?

I perfeclty miss read dev reply about some old issue, but today I tried to gather all info which related with driver update.

Then happend to find old bug report which I added, about custom prop up-date.

https://developer.blender.org/T74000

https://developer.blender.org/T63793

eg when bpy or script may set value for custom prop, then custom prop drive pose bone transform value.. if script set custom prop value, it not up-date correclty. but if we change value (the custom prop) manually,, blender up-date it correctly.

So dev reply , it is known issue, but use

object.update_tag(), then force to reflesh scene.

like C.scene.frame_set(C.scene.frame_current) can correclty update bone location.

there is no difference, if I add csutom prop for rig.data, or rig, about both case,, after set value.

like rig[“prop”] = 20 or irg.data[“prop”] = 20,

rig.update_tag() # it tell rig, to update all custom props of the rig.

then force scene up-date eg use C.scene.frame_set(C.scene.frame_current) # I suppose we can use another way to force up-date ,

Actually if I do not add rig.update_tag(), bone position not up-date, even though I use C.scene.frame_set(C.scene.frame_current)

But if I add rig.update_tag() after set value by bpy for driver csutom prop, then object.update_tag() work.

I think it may somehow related with custom prop (morph) up-date issues when they are deep hieralchy. you may check this too.

https://developer.blender.org/rB66af6e4f32148e254a3dc64e442b4b6f389259d8

[Thomas Larsson]

I haven’t noticed that any morphs failed to update yet, but that is definitely something that I have worried about all the time. It is good to know about the update_tag function.

I definitely don’t like the idea to use scripted expressions as long as it fits, and then switch to sum drivers. That will definitely mess up the code. To split the fin value into raw, sum and mlt parts is nice though. That should fix the problem with the facs expressions, if it updates correctly.

Another problem with the sum drivers is that they take a long (quadratic) time to load. More exactly, adding variables to the sum driver for a given by is proportional to

(the number of variables for this bone) * (the ackumulated number of variables for all other bones added before).

This is very obvious if you make an easy import with face units, expressions and visemes, and look at the output in the system console.

[Thomas Larsson]

I think I solved the problem for now, although it will resurface once the driving expression becomes too long The idea is to put the multiplier to the left. Say that after loading the facs units we have the expression

M*(a+b)

Looking at the right end, we see that it ends with a “)” and that the last variable is b, and we don’t have to worry about what is to the left of that. So we start adding new variables to the right, starting with c. After face expressions, we may end up with something like

M*(a+b+0.3*c+0.15*d)

‌

[engetudouiti]

As same as Thomas, I have not noticed miss up-date morphs with current test abouot new morph system, but I suppose it relate with some strange issue, about MHX did not up-date correctly. (though untill I confrim it, you seems up-date and solve most of issue?)

when we change raw slider value directly in UI, most of case driven props up-date corerctly, even though prop(raw) drive prop2(fin), prop2(fin) drive prop3(fin)……. when we manually input each raw values of morphs, these circulation are done in driver chain only. about the case up-date problem may not happen.

But if you need to set some prop value via script, like copy some value and paste to aother prop value, or function set prop value,, (IK FK switch etc?) without driver, and the prop drive another prop by blender driver, I suppose this up-date problem may happen.

I remeber, we solved this issue by change current UI mode, or as dev show, scene.frame_set(C.scene.frame_current) etc, then force target prop up-date. So if we will see same up-date issue, you might use the update_tag with those scene update, then may solve issue I hope.

I definitely don’t like the idea to use scripted expressions as long as it fits, and then switch to sum drivers. That will definitely mess up the code. To split the fin value into raw, sum and mlt parts is nice though. That should fix the problem with the facs expressions, if it updates correctly.

Actually I do not think we need to use the “Sum Values” . I did not check how actually Xin add interemediate props, then I saw “Sum values” type is used for G3 facial pose bone driver expression (to driven by many pose controller) so just tried to use “Sum values” as you did for pose bone transform props.

About pose bone tarnsform prop dirver expression, each (morphA * step value), ( morphB*step value) , (morphC*step value) are converted as new custom prop for bones. then if we do same thing, for facs untis, as bone transform property, we need to generate all intermedieate props for one rig custom props, or one rig.data custom props. I think it may work ,but add really many custom props for one rig or rig.data. so when we check custom prop, it really hard to check those I suppose. (though if user not check custom prop it may not problem)

So I may hope anyway you can find way,(you will need it) with your new format. M*(raw+a+b+0.3*c+0.15*d)

when M*( raw + a*0.359 + b*0.468 + c.0.479 + d*0.37 + e*0.2 + f*0.58 + g*0.65 +…) >> limit over

you convert it as M* (raw + (a*0.359 + b*0.468 + c.0.479) + (d*0.37 + e*0.2 + f*0.58 ) + (g*0.65 +…) >>

M*( raw + (m1) + (m2) + (m3)…) .. so you can multiple or input new variable as same as before, when user import new dsf. then if

This expample generate new inter m1 m2 m3,,, prop with each 3 pair “( )” so add on try to convert new prop when you add 3 new variables. I suppose, if you can make such function, devide expression as new props, and combile as new expression..

[engetudouiti]

Then If use same way to generate M1 M2 props as same as bone transform prop driver expression, , I suppose, if you can generate new pose bones to attach custom props, then can divide each expression middle props for each Morph.

(as you already do it for mix euler), about the case those bone can be perfectly hidden, then each bone represent each morphs driven sum part only, . it is only used to attach intermediate expression props.

like m1 = (a*0.359) m2 = ( b*0.468), m3 = (c.0.479)

It means use bone to represent each morph (dsf) drv-sum part as same as you use each drv-bone to attach middle props.

MorphA (fin) = (Multiple part) * ( MorphA(raw) + MorphA(drv-sum))

then when import new morph X, if it drive MorphA,(fin) as delta add, like morphX(fin) *0.965

, you add new custom prop for morphA_drv bone. as bones[“MorphA”][“MorphX”]

Of course if you can group those middle morphs for each Morph(fin), without use pose bones, it need not. but if you do not find clear group way, , the drv part expresison easy mess up… you may need to care how you name and generate those middle prop, like m1 = (a*0.359) m2 = ( b*0.468), m3 = (c.0.479) for each Fin morph.

[engetudouiti]

The difficulity is,, I think we can only use ID props to generate those expression props,,,

I do not think ID prop can generate kind of dict type. which work as hieralchical driver, … but if it work

we already have raw and fin prop, then add sum prop (as dict type) . then generate new middle expression prop,,

[morphA_sum][morphB], [morphA_sum][morphC] [morphA_sum][morphD] …… as same as we attach middle prop for each drv_bone.

then [morphA_sum][moprhB] driver expression should be morphB(fin) * step value, which included in MorphA drv_sum part.

===

The best way is ,request blender dev to input step value for each Variables , when we use “ “Sum Values””

at current it can use only for (A + B + C….) if we can set step value (multiple as strangth) for each Variable A, B, C,

then use “Sum with step value” make things really easy ^^; I feel, if Thomas request it in your report about expression limit issue, they may consider..to add new type Expression whch consider variable step values.

I am not good for the purpose.. (I often miss use word, then it make dev team easy reject ,, )

I suppose,, we may not see limit problem,, for a while,, so have good time,, untill you need to change expression. for Facs morphs controllers.. I know at current these things are just need for future problem. (if vendor offer such new expression product, which drive Facs Units etc,, though I suppose daz push it ^^;)

[Xin]

Thomas that problem with adding a lot of variables seems to be a poor implementation on Blender’s side, it happens with all drivers when you add a lot of variables, it’s not related to any particular type.

A workaround is shown in the attached .blend, which keeps the time under control.

[engetudouiti]

btw

I actually do not have strong view, “Custom function(python)” or “blender offer function” to solve this issue. Actually most important thing is, work or not. 2nd issue is peformance when we set slider, pose, and when we load those many complex morphs

Only thing I consider was structure of Driver Expression. for new morph system. , which use raw, and fin for all morph.dsf with same rule. as daz do it. Then I still do not decide which option I use as default.

To make more clever Expression generator,, I believe use Custom function can enhance more easy. at same time it can easy peformance issue and non Csutom function work beter, when we pose or change value of shape key. (I suppose so), but I know it take long time to generate only for default Facs morphs. (G 8.1) ,, and G3 face unit too.

I still not test much for Custom function,, but I think if we use custom function, there seems no problem already when we generate morph(raw), and morph(drv) and morph(fin) for each morph.dsf.

‌

‌

[Xin]

Possible workaround for Blender's performance issues with adding a lot of driver variables.

[engetudouiti]

Use “Sum”(non custom funciton) need to generate too many custom props for multiple step value with driver variable. , for each drv pose bone. it not happen if we use custom function. because we need not generate new custom props as for each variable * step value. Then the property count is matter to import morphs I suppose.

‌

[Xin]

You still have to store the variables and evaluate the same function. Just because they are hidden in Python doesn’t mean they aren’t there.

Blender itself recommends to use Simple Expression as much as you can both for performance reasons and for dependency issues. That’s because with Simple Expressions, inputs are made explicit to Blender. They are much safer and give better performance too as Blender can use multithreading with them.

I don’t see how trying to save a little time on coding (or on loading) to pay a heavy price on performance and potential dependency issues would be a good decision.

[engetudouiti]

I understand, it clear. so do not teach me same thing again. I said, property, which you generate to get value.

of course I know, when use custom function, those variable need to generate, but not add ID prop like current way.

Then you should not say I do not understand. I just offer way, to work better with my guess. actually your way have cause issue , so Thomas needed to return Custom function as default once.

blender recommend to use simple funciton. of course I know. at same time it describe, use custom function if it need. I do not against try to make all work without custom function.

if genearte such many custom props not cause issue, I simply like to use “non custom function”

‌

‌

[Xin]

That problem is just a loading performance problem, and can be sidestepped, as the attached .blend shows.

Look at what addons with a lot of drivers do:

The RBF Drivers addon, which has a lot of Drivers, manages to implement everything with Simple Expressions, creating a lot of drivers in the process: https://rbfdrivers.readthedocs.io/en/latest/user-guide/overview/ It explicitly tries to avoid custom functions for performance reasons.

Another thing: implementing a Simple Expression summation of the type S0 * M0 + S1 * M1 + …, etc. can be accomplished and made easier by implementing a class that manages the summation and the carry over to a new driver when the limit is reached. I don’t know if it’s necessary though.

I just don’t see any reason to use Python for sums and multiplications.

I’m glad that you are helping a lot engetudouiti, your advice is always well thought and useful. I just disagree with using Python functions.

[engetudouiti]

I know what you have described, and I have interensting RBF driver, I actually thought to recommend it, for all morphs too. if Thomas do not care to add new bones anymore. But I do not think it is good to rely other add on much, as same as auto-up-dater cause issue. only the add on author can solve issue, if it cause new problem.

Then you miss understand why I said about both option. even though you like non custom function, actually it have used to represent, the prop value * step value, long time, it is reason why Thomas need to use it. (of course we know custom function cause peformance issue) but to manage many controller, it is suggested and I simply like the way.

So you offer way not use custom function, and recommend it, I do not against it never.

At same time, without custom function, we needed to generate new props for all conotroler prop for each drv pose bone. I simply surprised when I confrim it, (though if it work well no problem for me)

unfortunately it seems cause peformance issue when import morphs. Then decide, which option is Thomas. so now Thomas try to work for both. Then I test both option. or may only try to use custom function, untill it will be change as new morph system.

then if you solve issue, it is really apreciate. but I definitely not hope you said me, “I do not understand custom function demerit”

‌

‌

[Xin]

As long as it doesn’t become a lot of work for Thomas, I agree that the more options there are for testing, the better.

Sorry if that came off aggressive, I just wanted to make sure you understood the reasons why simple expressions are favored. I didn’t know you already knew that.

[engetudouiti]

Thomas need to offer default option, which user can use it without problem (less problem) even though for beta... Then for some reason, (unexpected), Thomas need to set default option which use Custom function again (load morph issue etc). Though I do not know all detail.

I actually hoped to test with one option (non custom function) then did not think seriously ,peformance about loading morph. but MHX issue is new problem for me.

At same time now we have problem when we import each morph category,, for both option. easy mode is good to import all default morph as one category, but I suspect, maybe user may not hope to gather all default morph as one category.

To solve those things, we need to think good way, how add new variable and prop , with use current expression. Then it may need different way for both option. then it seems partially solved, but future we may see same problem (limit), then when use same way which used for pose controller morphs, we need to think how attach new props which represent each varialbe * step value.

you may notice most of all what I described here is try to use non custom function. I expect your new attached blend file may offer some hint or solve issue. (I do not think I can solve it with my knowledge)

Then I make this just to use non custom function, to make it easy today.. I do not like to request in right-clidk select though. (I like to talk with dev directly, or I often get reply which do not understand why I request it)

maybe some user may reply, you can use custom function, or you should not use such long expression etc.

https://blender.community/c/rightclickselect/zjhbbc/

it is just because to avoid custom function.

[Thomas Larsson]

Xin, adding the drivers to a temporary object first really improved loading time a lot. A lot!! However, now I have another problem. The pin and clear buttons have become much slower, although AFAIU the final drivers are exactly the same as before. The problem is in the function that updates drivers. It loops over all driver f-curves and overwrites the driver expression with the same string to force the driver to update. There has to be a faster and more elegant way to do it.

[Xin]

As engetudouiti pointed out above, there is a function that is much cleaner that does that job. See https://developer.blender.org/T74000

Documentation: https://docs.blender.org/api/current/bpy.types.ID.html#bpy.types.ID.update_tag

I did a quick test and changing to this:

def updateDrivers(rna):
  def updateRna(rna):
    rna.update_tag()

Seems to work fine. It can be cleaned up more I think: I believe it’s enough to just tag the object where the property is defined (the rig here) and you are done. So the entire updateDrivers() function could be removed.

---

Also, it might be a good idea to report the performance issue with creating driver variables to Blender (in fact, not so much creating them, but editing them, since creating them is very fast, it’s the editing of their attributes what causes the slow down). It’s silly that we have to create a temporary object to sidestep that.

In the end, I think Blender will end up overhauling the entire system with the node system that is planned for animations too (recently implemented in the latest release for geometry).

[Thomas Larsson]

Yup, simply calling update_tag() brings the buttoms up to speed again. Since the simple drivers now work as good as or better than the old custom drivers in every respect, the latter have been removed.

[engetudouiti]

You return default option without custom function. thanks. Then I suppose, now sum drivers always use armature properties. But raw prop are attached for object still even though these 2 type prop value are used in same driver expression.

I do not think we must need to make it so, at current it cause no problem for me. but set raw props for rig.data may cause new issue? I expect if use same data type to attach prop, which value used in same dirver may offer more peformance.

And If you divide, morph related property attached to rig.data and other props which store daz relatad infomation (like imported duf path etc) are attached to object may make things more simple just for UI too. I may not request but just hope to know if you think it cause problem or not. Though I actually now asking same thing to blender dev, in bug report (blender) which you recommend when attach custom prop for armature driver variables.

[engetudouiti]

Actually new morph system is really amazing. I sometimes see question why we can not import vendor expression etc.. because add on could not manage morph which change already improted morph (like default face unit)

Now I could import vendor facial expression morph. which need to change daz default face unit morph values. it work without problem. (I just worry, if there still remain, some driver expression will be overwritten without intentsion, = old import morph driver would break)

But at current I do not see this issue anymore. Thanks all your work Thomas.

‌

[engetudouiti]

Ah,, I found one issue ^^;

when I import those vendor expression as custom morph,,

add on now generate same face unit, which used for the expression as custom morph >category again.

It is not good..(though it still work)… (I suppose it is current problem is how install custom morph and not duplicate morph)

Fortunately, it not duplicate custom property, so it seems UI issue only.

=======
No actually if I import the package, it clear show, the limit issue. so same face unit morph are driven by many new facial expression,, like this.

The limit problem may come really easy,, when user import those package morph, which controll alrady import morphs, then generate new face shape (expression) this pic clear show problem

about this case the epxression cut out, and now it can not return zero. even though I set all raw prop as zero.

The Z morphs is product for G3, it mix use face unit, and pose bone directly.

But I suppose there will be new product for G8.1 Facs Unit,, and it main use Facs unit morphs. so same problem should happen (over-flow limit expression), without we slove issue for Property drive Property. as same as property drive pose bone. (it was sloved)

‌

[engetudouiti]

If we use Sum, for only one Face unit (fin) expression, we may need “a*step value”, “b*step value,”, c*step value >>>>w*step value as new custom props. Tough I suppose even though you generate all middle prop for one rig. or rig.data, it still work.

I may recommend,, 2 things first..

when the variable come to p , (a to p can represent 16 variable) , next variable use a1 to p1. then if it come to p1, next use a2 to p2.

and for raw value to use “r”,(or any without a to p as you like) then only use a to p for sum part variable.

if you make variable so,, (r (raw) and a to p, a1 to p1, a2 to p2) for morph driver expression, (prop drive prop)

full expression may change like this

morph(m) (full length) = r + a*st + b*st…..p*st + a1*st …… p1*st+ ……..a2*st + b2*st…p2*st + a3*st + b3*st + c3*sp)

why it may be useful,, if you will plan to generate new prop for each (a to p) section you can convert it as

r + (a*st…..p*st ) + (a1*st+ …… p1*st) +(a2*st;…p2*st.) +(a3*st + b3*st + c3*st) easy…

I suppose,, each (a to p) section middle prop can name as morph_0 , morph_1, morph_2, morph_s3 ….

(though I do not know you will generate those middle prop, what I suggested or not)

final expression can convert as morph(fin) = r + morph_0 + morph_1 + morph_2.

if you make it so,, 48 new prop turn as 3 new prop.

of course there should be many way how repersent and divide. it may change your format way for variable,the key concept is,, use alphabet only for each purpose, to divide each section clear,. so you can manage it when you will need it. by use alphabet.

[engetudouiti]

Btw can you generate those long full expression in log.txt, when you build driver expression?

Because even though it cause limit error, if there is right expression, user can correct it. (manually divide etc)

without right one, user need to check daz studio ERC. and compare each morph in bleder, eg i do not know real expression about attached pic, If there is generated expression strings I can test , with divide props etc.. or test generate dummy bone for each controller (face unit), and attach A*step value to Z*step value props, And use it with Sum etc,, (can real test it work or not with already generated morphs)

[Thomas Larsson]

Sigh. I suspected that the driver expression would overflow eventually, but I didn’t expect that it would happen so soon.

[engetudouiti]

At current it only happen when user test import some morph set product, as full.

I understand it is perfectly future up-date things, (not expect all work so soon)

I actually be excite, to seet, we can import some hieralchycial morphs already without you do not manage case one by one.

Before I never plan to import Z morphs. (because I know add on should cause erroer) . The product morph simply change Face unit values, (default morphs which controll face bones) so, I could easy expect, it not work.

but now we can import them and work. (I did not see heavy pefromance issue)

only pity things, import custom morph still cause some UI problem, and maybe we need error handling, if it show limit. (eg,, show message, then pass it) , I happend to notice, it when I set zero morph. at first I afraid if it cause new issu,e but it work as expected. so only problem is expression was broken (cut with limit) and it not cause erroer. (because the expression can still get value, . from input variables, even though it not represetn all formula^^;)

‌

[engetudouiti]

Today I deep test animation node plug in (first experience),, it already offer node we can manage this problem easy. might you test it ?

it is simply interesting enough even though not use it for add on. (and free!! ^^b)

https://docs.animation-nodes.com/

‌

Animation node can do most of things which we do with driver and variables.. , by directly set each value and data as node. it have many unique nodes. eg list float node can gather values as list, then can use list sum.

in this pic, I do not set any driver, but when I change each custom prop, with multiple value, then gather them as list, it actually circulate all and pose bone rotate. (though there is no good out-put which manage pose bone easy,, I need to use object output node, and set attribute. I could not find armature output,, so use Object.data …,)

But unfortunately,, even though if it work well with real character and 100 over hieralchical morphs, shape keys, the add on is not blender official so I understand you may not plant to use it. so we may need another clever way to manage long expression.

[engetudouiti]

Though I do not plan to use animation node for all driver of this add on, as real solution I understand it can not use for Daz importer. then after all we need to divide the long expression. (we can not avoid it)

the way are

A make new intermediate props for each “morph(fin) * step value “ (sum part) . then use Sum values.

B make new intermediate prop for each section, deivide by variable count. like

r + (a*st…..p*st ) + (a1*st+ …… p1*st) +(a2*st;…p2*st.) +(a3*st + b3*st + c3*st)

I know both way may show pro and cons,

A need more custom props (eg If I import z morph package (20 morphs) which controll G3 20 face units,, 20 * 20 new custom prop need (aproximately), but there is no bone to group them, so we need to generate those new ID prop without group. (only name can be used and it should be long. like

[“eye squint right : z FI anglry_sum”]…….

B can limit prop count, and I suppose, the intermediate prop can name, more simple like “eye squint right_s1” s1 s2 represent section number for sum part intermediate props. but to auto generate such props and devide by script is more complex. than A ,

eg when we remove one morph, it need to serch intermediate props, s1 to sn, and only change the expression.

If do not bother huge props count, I may recommend to use A. (how group it or non group may depend Thomas, eg add new dummy bones with each face unit name, and attach those intermiedeate props. can gorup by each face unit.. (though prop count never change)

[engetudouiti]

btw I could confirm current limit is 256 unfortunately. it seems temporally change as 512, but after all the commit was reverted. (do not know reason) I feel 256 is really easy overflow for G8.1 face unit controller.

G3 is lucky because daz offer expression not directly use G3 face unit. but adjust bone directly as same as G3face unit.

about G8.1. most of expression may offered only controll facs units. (so facs unit was important to work ERC correctly) if there are some new package expression, it easy break current actor.

[engetudouiti]

I think, prop attached to rig.data seems not cause issue. (though I feel. there still some problem to import morphs) then close this.

maybe ask each problem relate morphs separately.