transfer animation to shapekeys ?

[Alessandro Padovani]

daz studio 4.15.0.30, blender 3.3.0, diffeomorphic 1.6.2.1196

This refers to a daz discussion linked below. Basically once we convert morphs to shape keys there’s no way to transfer the animation to the shapekeys, or to load a daz animation on a figure with shapekeys. Unless there’s some tools that I miss that’s entirely possible.

https://www.daz3d.com/forums/discussion/592141/

What I was able to do by hand is to convert to shapekeys then cut and paste the animation timeline from the morphs to the relative shapekeys, so this should be possible to automate.

steps:

1. load the daz animation on the genesis figure
2. convert morphs to shape keys
3. cut and paste the timeline from the morphs to the shapekeys
4. delete or disable the original rig

As a test I’m including a simple “speak“ animation for g8f using only the “mouth open” morph.

Let us know what you think.

‌

[HALO RECLAIMER]

Hi 

My own tests confirm that it is possible to manually transfer the phoneme& fa cial shape keys ,as morph based animation, to the DAZ mesh and retain the facial animation, even after deleting replacing the Daz rig with an ARP  body control rig.

‌

However it requires individually matching the phonemes and morphs by name which not practical without some manner to script the copying based on the matching names.

‌

the way Blender works it may be possible  to Py script this task  separately without even building it into Diffeo itself.

[Alessandro Padovani]

Yes copying and pasting the individual tracks is a pita that’s why I’m asking Thomas if this can be automated.

As for loading the daz animations to shapekeys from a duf file, I guess we have to rename the shapekeys with the daz id rather than the daz label, this way we match the names in the duf file. We could have an option when converting morphs to shapekeys to use the label or the id for the shapekey name.

[HALO RECLAIMER]

Glad to know that this is all down to name matching so no actual new software needs to be invented to achieve this.

‌

I know other animators like Bennie Woodelle could benefit from this, as he uses alternate rigs as well

after import Via Diffeo.

‌

Thanks

[Thomas Larsson]

I implemented a transfer tool in the last commit. It is located right under Convert Morphs To Shapekeys button in the Advanced Setup > Morphs section.

‌

The tool transfers F-curves of the active rig action to all child meshes, if the child has matching shapekeys. All F-curves that have been transferred are then removed from the rig action, to avoid duplication.

Perhaps it would be better with a batch version of the tool, which can transfer multiple actions at once.

[HALO RECLAIMER]

HI first thanks

and forgive my bit bucket ignorance but where do i Find this latest commit??

‌

Also as this a feature for  specifically for facial animation imported from various Daz sources(mimic,Anilip.etc), I dont see how we would need to batch convert multiple facial performances in one go

[HALO RECLAIMER]

HI , I found the commit

Updated both both the Blender and the Daz scripts.

‌

Sadly this latest commit completely breaks the ability to “import action” or poses with error

“no mesh found”

‌

Blender version 3.22

Daz studio version 4.12.086 windows 10.

‌

Reverting back to previous version of diffeo restores ability to import action/poses

[Alessandro Padovani]

Commit 82e95e7 works fine here. @Halo, if it doesn’t work for you please describe the exact steps.

my steps:

1. import g8f with face units
2. import a daz animation using the face units
3. convert morphs to shapekeys
4. transfer animation to shapekeys

@Thomas, one purpose of this request would be to use a custom rig for the figure and be able to import the daz facial animation on shapekeys. For this it is necessary that the tools can work on non-daz figures. And probably to name the shapekeys after the id rather than the label, so that the daz animation can be loaded from the duf file with matching names. When converting morphs to shapekeys we may have an option to use ids or labels for the shapekey names.

‌

[HALO RECLAIMER]

Hi your steps are my exact steps

‌

I am using a basic talk sequence created with

32 bit mimic basic

‌

if i revert to previous diffeo version the daz .duf file loads ( via import action) and the figures head speaks

‌

with the new commit “import action” fails with the error mention before.

‌

Honestly the commit/bit bucked system is new and confusing to me

‌

is it possible to link me directly to the new commit so I know I am grabbing the correct one?

thanks

[Alessandro Padovani]

To get the latest commit go to download > download repository.

If possible please upload your test animation so we’re in the same boat. What morphs does mimic use ? How do you get/install mimic because I can’t find it in the 64 bit distribution.

edit. I see mimic only works with the 32 bit version. I’d suggest you to use papagayo instead that does the same thing and it’s free.

https://www.autolipsync-o-tizer.com/

[Thomas Larsson]

The tool assumes that the mesh is parented to the rig, which is normally the case if you use the other plugin tools. I could change the interface so you select both the rig (active object) and the target meshes (selected objects). Then the parent relationship isn’t necessary, and you can better control to which meshes the animation is transferred to.

[Alessandro Padovani]

That the mesh is parented to the rig is quite normal in blender, even outside daz figures, so I don’t think it is necessary to change the interface.

[HALO RECLAIMER]

Here are my steps.

loaded fresh g8 male in 32 bit Daz studio 4.20

created basic lip sync with mimic

saved as animation pose file .duf.

Loaded base g8 male in 64DS  bit and dragged .duf file onto him to be sure and he speaks .

Loaded same G8 male into blender with all face using using new commit.

tried to load animation file and fail with error ( no mesh found)

import action  still working perfect  over on 10 year old toshiba laptop

with previous build of diffeo and Blender 3.22

‌

Tested on old diffeo  on Blender 3.3 ( stable release) worked fine

Update Blender 3.3 to latest commit…”import action” fails

reverted Blender 3.3 and “import action” works perfect again

DAZ mimic .duf file attached for testing

https://drive.google.com/file/d/1FCY571im9CXjviXdD9ODnl8DMn1wCvqs/view?usp=sharing

‌

‌

[HALO RECLAIMER]

OK 

used the free lipsyncotizer app to create MOHO file

that still works fine in new commit

https://www.autolipsync-o-tizer.com/

‌

frustratingly  The transfer animation to shape key button throws a python error:

‌

‌

Python: Traceback (most recent call last):

File "C:\Users\Dhul-Quarnain\AppData\Roaming\Blender Foundation\Blender\3.2\scripts\addons\Diffeomorphic-import_daz-82e95e71c85e\error.py", line 207, in execute

self.run(context)

File "C:\Users\Dhul-Quarnain\AppData\Roaming\Blender Foundation\Blender\3.2\scripts\addons\Diffeomorphic-import_daz-82e95e71c85e\morphing.py", line 2985, in run

self.copyFcurve(fcurig, fcu)

File "C:\Users\Dhul-Quarnain\AppData\Roaming\Blender Foundation\Blender\3.2\scripts\addons\Diffeomorphic-import_daz-82e95e71c85e\morphing.py", line 2992, in copyFcurve

fcu2.keyframe_points.clear()

AttributeError: 'bpy_prop_collection' object has no attribute 'clear'

‌

‌

‌

‌

‌

‌

[Thomas Larsson]

Made some fixes in last commit, but must run now.

[Alessandro Padovani]

@Halo your mimic animation works perfect here. It’s the same in daz studio and blender. I believe you’re using the default settings to load an action, that without a scanned database may make little sense. The latest commit by Thomas should solve the issue, or try with the settings below.

Please note that your mimic animation uses face units and visemes, so you have to load both. Also it uses the head and eyes bones, that will not be transferred to shapekeys.

Let me know if this works fine for you.

steps:

1. import g8f or g8m, with face units and visemes, make all bones posable
2. import the mimic animation with the settings below
3. convert morphs to shapekeys
4. transfer animation to shapekeys

‌

[HALO RECLAIMER]

OK that got the latest commit to start accepting mimic pose files file again at least 

‌

but still the transfer animation to shape key tossed the same error I posted above.

‌

Time check Blender market for some other shape key baking method, but thanks for all your help though

[Alessandro Padovani]

@Thomas Commit 834aa31 works fine. Now we can use the morph names for the shapekeys, that are the same as the duf file. What we can’t do is to load the duf animation to the shapekeys, that should be possible now perhaps with a new option in import action. This will allow to use a custom rig and be able to load facial animations.

import action:

• new option: “transfer animation to shapekeys“
• description: “Load the daz animation to shapekeys, requires the morphs to be converted to shapekeys first. Please note that keyframes without a shapekey will be ignored.“

steps:

1. import g8f with face units and visemes
2. convert morphs to shapekeys
3. delete the rig, this leaves only the mesh with the shapekeys
4. import action with the new “transfer animation to shapekeys“ option. Please note that the action may contain keyframes that don’t have a converted shapekey, in this case the keyframe should be ignored with this option.

‌

[Alessandro Padovani]

@Halo, well at least you got the mimic animation to work in blender that’s a good step. Now “transfer to shapekeys“ works perfect here. I’m using blender 3.3 don’t know if there’s any difference with 3.2 I guess not. What are your exact steps ?

my steps (you’re fine on step 1,2 and fail 3,4):

1. import g8f with face units and visemes, make all bones posable
2. import the mimic animation
3. convert morphs to shapekeys: you have to select the g8f mesh and the morphs you want to convert, after that you can see the shapekeys in the blender panel
4. transfer animation to shapekeys

‌

[HALO RECLAIMER]

instead selecting all of the morphs

in step 3,

I hand picked just a few  visemes ( AH,TH etc)

they then  appear in the shape keys panel.

‌

The transfer animation button still fails ,same error message as before.

again thanks for your attempts

must be some unknown variable on my end that is going to force me to find another solution.

‌

‌

thanks

[Alessandro Padovani]

Don’t give up, it takes time to fix a new feature.

It must be some morphs that diffeomorphic doesn’t like then. What morphs do you import other than face units and visemes ? Did you try the import options I posted above ?

Please make a detailed step by step of what you do with pictures so we can do exactly the same. Or a video is fine too.

[HALO RECLAIMER]

I have already tried the bare minimum visemes for lip movement only

eveythings works perfectly from a mimic file to a papagayo .dat file from lipsyncotizer.

‌

It falls apart at the transfer animation stage

‌

on a bright note I can still create ARP body rigs with a full sets of Daz facial Blend shapes in the head and manually change face expressions during the action scenes where I really need the ARP retargeter& rigs

‌

I would just have to keep a second copy of each Character with his original Daz/dfffeo /rigify  rig for extend speaking shots using mimic etc.

[HALO RECLAIMER]

Here is a video of my steps

those purple third party shape keys are the possible culprits

‌

‌

https://drive.google.com/file/d/1Lnr6sUygwGVfPxa6F5IznzO9z0Ie6rIr/view?usp=sharing

[Thomas Larsson]

The crash should be gone now. The api had changed between Blender 3.2 and 3.3, and I was relying on a new feature.

[Thomas Larsson]

Now Import Action has an option to import morphs to shapekeys instead of rig properties. That option also disables everything that has to do with missing morphs, since morphs are loaded as rig properties.

[Alessandro Padovani]

@Halo, in your video I see you import facs instead of the face units. The mimic animations use face units and visemes, not facs. That is, facs are for facs animations that’s iphone mocaps typically for G81, not for mimic. When converting shapekeys you can use the type selector. Also Thomas fixed the error with blender 3.2, let us know how this works.

‌

@Thomas, I’m not sure commit b67e64e works fine. That is, with the new option, import action should be available also for non-daz figures, since once we use a custom rig the daz rig is not there anymore. If I delete the rig and keep the mesh with the shapekeys then import action is not available. Unless I miss something. May be we can do a separate tool for “import action to shapekeys“, instead of the new option ?

‌

[HALO RECLAIMER]

Thanks guys

it is all working now .

In both Blender 3.22 and 3.3

This is everything I was looking for to combine Daz

native facial animation with Autorig pro body rigs.

I have even sorted out which morphs to load

(the base visemes and expressions only)

and figured out how to replace the current morph based face animation with a new import from mimic or lipsynctotizer.

‌

I was nearly about to spend $80 USD on this add-on

https://blendermarket.com/products/faceit

‌

You gentlemen have not only saved me money

but this also completes my migration from Reallusion CC3 back to Daz diffeo for Character animation work

‌

[Thomas Larsson]

Alessandro, if I remember correctly the posing buttons are present as long as the active object is an armature. Any armature, not necessarily a daz one. They are hidden if the active object is a mesh. But I have to check that later.

[Alessandro Padovani]

@Halo, glad that it works now. I’m asking Thomas if we can get to load the daz animation directly on the custom rig, otherwise you have to keep both on the scene.

@Thomas, I replaced the daz rig with a single bone and the posing panel shows up that’s ok. But then import action does nothing with the “load to shapekeys“ option, even if the shapekeys are generated in the mesh with the morph names.

steps:

1. import g8f with face units and visemes
2. convert morphs to shapekeys, with morph names
3. replace the daz rig with a single bone, that represents a custom rig
4. load the mimic animation, with the “load to shapekeys“ option

‌

[Thomas Larsson]

Should work now, both if the armature or the mesh are not recognized.

[Alessandro Padovani]

Commit 260c2a0 works great here.

There’s issues with facs that are reported in #1180 and I replied there my point of view. I don’t think this is intended to be used with facs, so if @Halo has nothing to add I’ll mark as resolved.

[HALO RECLAIMER]

Not really sure what this last commit actually does differently but..yes  for me this matter is resolved.

‌

Thanks

[Alessandro Padovani]

The last commit allows to load a mimic animation directly on the custom rig, that wasn't possible before.

Marking as resolved.

[bouich jules]

Hmmm interesting, so i can have a figure without the daz rig but with daz shapekey into it, and with this new function i can transfer animation to this rig?

correct alessandro?

[HALO RECLAIMER]

Yes Facial animation & lipsync created with one of the Daz native tools( mimic 32bit,Face mojo,Anilip2)