BVH Retargetting still failing **request for bandai bhv**

[GeneralProtectionFault]

Blender 4.0.2

Import Daz: 1.7.4.2045

BVH: 2.2.0014

‌

So I saw this issue here:

https://bitbucket.org/Diffeomorphic/import_daz/issues/1958/mixamo-to-genesis-9-not-working-bvh

…which is marked as resolved, but I’m still seeing the error. I actually tried in both Blender 3.6 & 4.0.2 (same plugin versions).

I’m just testing w/ the default G9 figure, and random bvh files from this repo:

https://github.com/BandaiNamcoResearchInc/Bandai-Namco-Research-Motiondataset

‌

I easy import the G9 base figure, click on the armature as per the blog post, click Load And Retarget, and I get an error.

In Blender 4, I get this:

File dataset-1_dash_tired_001 24
Name Y_dashtired001 <bpy_struct, Object("BvhRig") at 0x000001E76499B908>
Identifying rig
Using target armature Genesis 9.
Identifying rig
Hidden Hips
*** BVH Retargeter Error ***
Hip bone must have children: joint_Root
For corrective actions see:
http://diffeomorphic.blogspot.com/p/bvh-retargeter.html

…and if I try in 3.6, I get this:

File dataset-1_byebye_feminine_001 29
Name Y_byebyefeminine001 <bpy_struct, Object("BvhRig") at 0x000002173875F908>
Identifying rig
Using target armature Genesis 9.
Identifying rig
*** BVH Retargeter Error ***
Upper arm Head has no children
For corrective actions see:
http://diffeomorphic.blogspot.com/p/bvh-retargeter.html

‌

[Alessandro Padovani]

importer 1.7.4.2045, bhv 2.2.0014

Works fine here, apart some expected sliding due to proportion differences. I’m including a test animation converted from mixamo.

steps:

1. import a G9 figure, I used Kat
2. load and retarget

‌

[Alessandro Padovani]

mixamo animation, 72 frames

[Alessandro Padovani]

update. center animation bug.

Nope, it’s the “center animation“ option causing the python error, that I didn’t use in my test above. Apart the error, the animation works fine anyway.

‌

[Thomas Larsson]

The problem is that the rest pose in the bvh files is weird, and because of that the automatic identification of the bone hierarchy fails. You can see the rest pose by firstLoad BVH File and then turning on Rest Pose. A similar problem happens for the ACCAD male, see https://bitbucket.org/Diffeomorphic/retarget_bvh/wiki/T_Pose. This can be handled by added custom information for this type of BVH file. Will look into that later today.

A nice repo with bvh motions, btw.

‌

[Alessandro Padovani]

update. bandai bhv error.

The badai bhv doesn’t load fine and gives errors, but this is expected since there’s no retargeting option for that, that’s not a mixamo file. Unless Thomas wants to add support for bandai too. Example included.

If we use “load bvh“ instead of “load and retarget“ then the animation is loaded fine without errors.

This is a feature request not a bug.

‌

[Alessandro Padovani]

bandai animation, 210 frames

[Alessandro Padovani]

feature request for bandai bvh

[Thomas Larsson]

Implemented in last commit. Turn off Auto Source and set both Source Rig and TPose Source to Bandai.

‌

[Alessandro Padovani]

bvh ebba463.

The bandai test file doesn’t work fine, the left arm doesn’t animate correctly compared to the original.

note. The “center animation“ option still gives the “missing hip“ error.

steps:

1. load the bandai test file
2. load and retarget to G9, with bandai settings, I used Kat as G9 figure

‌

[GeneralProtectionFault]

Well shucks, didn’t realize the file “standard” wasn’t standard, thanks Thomas for working on it right off the bat .

‌

One thing I’m not quite clear on, Mixamo currently appears to just be fbx (not bvh). Is there a standard workflow to convert them and/or import them?

I noticed there is some existing code to convert here: https://github.com/DeepMotionEditing/deep-motion-editing/blob/master/blender_rendering/utils/fbx2bvh.py

…but not sure what the accepted way to import Mixamo is.

[Alessandro Padovani]

For mixamo you can import as fbx then save as bvh with blender.

As for “standard“, bvh is a standard for animation data but not for armatures, so for every different source Thomas has to add a new profile. Or you can retarget by hand using any retargeting tool.

[Thomas Larsson]

The center animation bug is gone. And you don’t have to specify the source rig and rest pose anymore, that is done automatically.

I can confirm the strange behaviour of the left arm in some cases. It seems to be caused by rotation limits. If I disable locks and limits the animation is much better. I’m not sure why retargeting tends to create poses that are outside limits in this case.

[GeneralProtectionFault]

Nice, I can confirm it autodetects nicely, and yeah also that arm/wrist stuff is weird. I don’t know if this helps any, but I tried just loading one of them (not load & retarget), and got a most interesting armature

‌

[GeneralProtectionFault]

Ok trying to get some diagnostic info here… As I’m sure is obvious, the screenshot above happens on some of the Bandai bvh files, but not all.

I’m going to use the 1st two in the 1st dataset as comparison:

• dataset-1_bow_active_001.bvh
• dataset-1_bow_angry_001.bvh

This works out because bow_active exhibits the problem, but bow_angry does not. bow_active looks like the screenshot in my last post, and here is bow_active:

So both files show that rather large root bone, I suppose that’s not a big deal.

It seems to have something to do w/ the OFFSET. I did a diff between the 2 files, and if you disregard the MOTION section, they are identical with the exception of the OOFSET on the Hips (I imagine this translation from the offset on the root, which is 0,0,0--maybe is the cause of the armature being translated all scrooged):

Another thing--BOTH files change orientation when going into EDIT mode (and it looks like this does not happen with the Mixamo file). Here is the same armature in EDIT mode:

‌

So, reading the specification a bit, that OFFSET is the X, Y & Z rotation from the parent. It looks like they (maybe) all have ~90 for the Y, and that EDIT mode here is reflecting that, and presumably the -452 Z as well. Perhaps having that nutty -452 Z rotation is screwing everything up. I don’t know why that would be there unless it gets into that quaternion double cover stuff which makes my nose bleed .

Maybe that’s wrong or perhaps if a rotation is more than |360|, then it could be corrected.

Anyhoo, not a veteran here, so forgive me if this ain’t helping xD, but seems pretty sus.

‌

[GeneralProtectionFault]

Ok so I tried zero-ing out that offset and rotation to test. It does stop the armature from spawning in the absurd positions/edit mode/etc…but it does NOT fix the bug Allesandro pointed out with the left hand.

However, I played around with it, and it looks like that bug persists using the DAZ rig, and MHX, but it’s different for the Rigify rig. It’s not right , but different! This is the unmodified bvh on all 3:

‌

‌

[Alessandro Padovani]

bvh c12dd5f.

@Thomas, disabling limits is somewhat better but doesn’t resolve the issue, I believe there’s some confusion when converting rotations. If we look at the first frame in the bandai example, at the collar bones, the original bvh gets the x axes always forward in the same direction, while the conversion gets the x axes in different directions by 90 degrees, and I suppose this is why the left arm behaves odd.

Let us know if this helps.

‌

[Alessandro Padovani]

bug. about frame zero.

I see that frame zero is used to store the rest pose, that’s not in the original animation. This is both unnecessary and harmful. In blender we get the rest pose by entering edit mode, or by using “rest position“ in the armature panel. Furthermore, if we use the rest pose at frame zero, we may get a wrong interpolation with frame one, depending on the first animation pose, especially with euler but it is also possible with quaternion if the rotation range to interpolate is wide.

bvh 4f9d98c doesn’t fix the axes issue.

[Thomas Larsson]

The pose at frame 0 is not the rest pose but the T-pose. The plugin puts both the target and source rigs in the same T-pose to calibrate the difference between the two. Rest pose is what you get if you load a bvh animation and go into edit mode.

To a first approximation, the target bones are put in the same pose in world space as the source bones. This is how the old bvh retargeter that was bundled with Blender used to work. However, this introduces unwanted twisting if the rest poses are different. By putting both rigs in a well-defined T-pose and noting the difference, the plugin can compensate for this twisting. This works well for reasonable rest poses (T-pose, A-pose, I-pose), but there are problems if the rest pose is too weird (Accad male, Endorphin, Bandai).

‌

[Thomas Larsson]

https://bitbucket.org/Diffeomorphic/retarget_bvh/wiki/How retargeting works

[Thomas Larsson]

The mhx plugin has a button called “Animation > Remove Frame 0” which gets rid of the extra T-pose. For some reason it ended up in the wrong plugin when I moved the mhx-specific tools from the bvh retargeter.

[Alessandro Padovani]

Thank you for the fast reply and the explanation. I see I made some confusion about the rest pose, the concept is the same though, an extra pose on frame zero may introduce interpolation errors compared to the original animation. Of course we can remove frame zero by hand in the timeline, so if this is necessary for retargeting then it’s not a bug. My bad.

I had a look at the article you pointed, and it is right we have to copy rotations in world space for retargeting. The thing is, it is actually done wrong. If we use constraints in world space on the left arm then the animation is fine, while the retargeting is bad so I guess it doesn’t copy in world space.

Let us know what you think and if this helps.

‌

[Thomas Larsson]

The solution was so simple. The theory is correct, but the Bandai T-pose was wrong; the left shoulder bone needed an extra 90 degree rotation around the Y axis. With that correction, the bvh file is correctly retargeted and the shoulder and arm bones are not twisted.

‌

[Alessandro Padovani]

bvh 98638fb.

Wonderful, works fine here thank you for the fix. If GPF has nothing to add we can close as resolved.

[GeneralProtectionFault]

Yeah great stuff, I haven’t been able to break it

I think we’re good, thanks much!