1. Thomas Larsson Avatar Thomas Larsson
  2. Diffeomorphic
  3. import_daz
  4. Issues

BVH retarget some small issues.

Issue #2033 resolved
bouich jules created an issue

Hi Thomas,

A small bug i discovered, is when i try retarget my UE to G9 fingers are not redirected

‌

‌

Also quick question please why does the clavicle are like this in G9 but not in G2, in G2 the retarget is perfect!

UE FBX animation i n attachment

Thank you so much!

Comments (32)

  2. Thomas Larsson repo owner

    I have actually never encountered a finger animation before, so this is an interesting test case. When we import the fbx file with the built-in fbx importer, each finger has five bones; two extra bones are added at the end of the bone chain. The problem is that the third bone (i.e. the last real bone) is oriented in the wrong direction.

    ‌

  4. bouich jules reporter

    Thank you Thomas, please if we can make sure, that all UE4 skeletton ( third bone oriented wrong ) are like this, not only my UE4 FBX.

    So we can make sure the fix apply for everybody who use UE4 skeletton.

    ‌

  5. Thomas Larsson repo owner

    As a workaround the plugin now ignores the third finger links for UE. The fingers still don’t look good however, especially the first thumb bone is misplaced. It’s going to take a while to figure out how to fix that.

    I also changed the Clear Bones tool. It now only clears the pose at the current frame, and shifts the poses at other frames. So the shoulders still move.

  7. Alessandro Padovani

    @Thomas, the issue with fingers is due to leaf bones, that is, fbx only has joints (aka pivots) it doesn’t have bones, so the bone orientation can’t be exported for the last in the chain (aka leaf bones). To solve this issue some exporters allow to export a extra pivot for leaf bones so to have the end point, but this extra pivot will be imported as an extra bone unless you ignore it.

    I guess this is what happens in the finger example by Bouich he exported/imported multiple times with extra pivots for leaf bones.

    p.s. You could include an option to ignore leaf bones, and use them only for the bone orientation, this is what most importers do. Unless this is specific for UE in that case you can also refer to the UE standard bones to detect the extra leaves.

  8. Thomas Larsson repo owner

    I think the finger animation works now.

    The extra leaf bones are removed from the fbx skeleton before the animation is exported to a bvh file. Also, the last real bone is aligned with its parent, so all fingers links point in the same direction. This lets us retarget the last finger links too.

    To compare the retargeted animation with the original one, I did the following:

    1. Disable locks and limits from the daz rig. The plugin respects locks so keeping them may change the pose.
    2. Load BVH or FBX file.
    3. Retarget Selected to Active.
    4. With the fbx rig selected, Center Animation to move it the same place as the daz rig.
    5. The fbx rig isn’t in T-pose at frame 0. To fix that, go to frame 0 and press Put in T-pose (either Source or Target).
    6. We can now compare the poses in the new Matrix panel. Go to some random frame and select the thumb bones from both armatures. The panel shows the location and the direction of the local Y-axis in world space. We see that the Y-axes point in the same directions ,which is what we want retargeting to do.

    ‌

  9. Thomas Larsson repo owner

    I removed the option to ignore finger animations, since it doesn’t make sense to me anymore.

  11. Alessandro Padovani

    bvh 2.2.0027

    Seems to work fine here, the fingers are straight in rest pose.

    steps:

    1. import G9
    2. load and retarget the fbx animation

    ‌

    Loading the fbx file alone has some issues, namely the rig is misplaced from the root, and “thumb_03” gets an odd angle. But these seem to be fixed when we retarget to G9.

    steps:

    1. load bvh or fbx file

    ‌

    Importing the fbx with blender doesn’t get the misplaced root, of course in blender the bones are all z-up unless we auto-connect, since fbx only gets joints and doesn’t get bones, so this is correct. Also there’s a lot of extra leaf bones so the original fbx file is not so good to start with.

    note. important. @Thomas, the blender fbx importer is way faster, so you may want to use it then retarget. I mean if it is usable by script.

    ‌

  12. Thomas Larsson repo owner

    The problem only happened with mhx but, like Alessandro, I only tested with the plain G9 rig. To calculate the reference T-pose the plugin loops over all bones, relying on the fact that parents come before children. However, in mhx the fingers don’t belong to the same bone chain as the fk bones (the fk chain ends with the fk hands). Hence the fingers were T-posed before the hand which messed things up.

    This is avoided in the last commits, but there are still problems, especially with the thumbs.

  13. Alessandro Padovani

    bvh 2.2.0030

    The root and thumb issues when we “load bvh or fbx file“ are still there, as noted above.

  14. Alessandro Padovani

    note. important. about leaf bones.

    @Thomas, I want to stress that the test file provided by Bouich is faulted to start with. That is, usually extra leaf bones are provided to mark the end points of the last bones in the chain. In the file provided by Bouich the extra leaf bones are in the wrong position, plus there’s a lot of them, this is something that Bouich or others messed up with multiple import/export options, but this is not usually the case. Then you may want to provide options.

    • use leaf bones as end points: this is usually the case, the leaf bones are used as end points for the last bones in the chain.
    • ignore leaf bones: if we don’t want to use leaf bones, then we can extrapolate the end points as the actual implementation does, in an attempt to fix bad leaf bones.

    ‌

  15. Thomas Larsson repo owner

    The extra leaf bones are not really a problem, since they are just ignored during retargeting. Aligning the real leaves with their parents might actually be a bad idea, at least unless the corresponding change is done to the action.

    Anyway, it was a long time since I seriously worked with the retargeter, and the code is quite opaque. I’m cleaning up the code and it will hopefully be possible to retarget fbx files directly, without taking the detour over bvh. However, this is not ready to be released yet.

  16. Alessandro Padovani

    If it helps. Blender can already load bvh/fbx very fast so you could use blender for that. Then “retarget selected to active“ could be used to retarget the bvh/fbx imported by blender. Then “load and retarget“ could use “retarget selected to active“ and delete the bvh/fbx as final step.

    note. important. about retargeting speed. Please note that retargeting can be a O(1) operation, that is, independent from the animation length, this is important to avoid long computation times for long animations. To retarget a bvh/fbx to a genesis armature we just need to adjust the genesis rest pose to fit the source bvh/fbx, then copy the animation. Though this also means that animations having different rest poses can’t be mixed together, so we may have an option for “fast” or “slow“ retargeting, meaning “fast” can only load multiple animations from the same rig type.

    note. about retargeting scale/location keyframes. We could have options to retarget scale and location keyframes from bvh/fbx, since mocaps don’t use scale and location anyway, and scale could be a nuisance if we normalize the armature. Or you could just limit retargeting to rotation to simplify things, that should work fine enough in the common case.

  17. Alessandro Padovani

    note. advanced. ik retargeting. In my own rigs I use constraints to retarget a source rig directly to ik targets, rather than fk bones. This has the advantage that hands/feet can never slide, and also figures with different proportions tend to retargeted better. But in my rigs I don’t use ik/fk snapping they always work in ik mode, so this may be more complicated for you, but something you may want to consider.

  18. Thomas Larsson repo owner

    In the last version we can choose to retarget directly from FBX. We can also use Blender’s built-in BVH importer, but the results differ (and are worse) than using the my own import script. I don’t know why. The options are in the Options panel.

    Changing the rest pose is of course possible, but it is not what this addon is about. And it may not be an option if you want to retarget to complex rigs such as MHX or Rigify.

    The plugin only retargets rotations, except for the hip bone which is also translated. Translations are automatically scaled to match the length of the thigh bone (thigh bend + twist for G8). The rationale is that pretty much every relevant character has a thigh, and this makes the stride right.

  19. Alessandro Padovani

    bvh 4.1.0033

    I can confirm there’s some misalignment between the blender bhv and the script. No idea why and who’s wrong. May be it’s just a precision issue importing the bvh values.

    steps:

    1. import a bvh animation with blender bvh or script bvh

    ‌

  20. Alessandro Padovani

    update.

    Comparing the imported values we see that the quaternion conversion is different. Plus blender also keyframes the bones location while the script doesn’t.

    ‌

  21. Thomas Larsson repo owner

    The plugin only imports location keys for the hip, because that is all that is needed for retargeting.

    All bones in the plugin have roll angle = 0, whereas the roll angles for the builtin bvh are nonzero and different for different bones. That affects the local rotation, since it is relative to the rest pose. To compare the pose, we can use Center Animation on both armatures which should put them in the same place.

  22. Alessandro Padovani

    Roll at zero is not a prerequisite in bvh, and it is also bad for genesis figures since this way you don’t use the twist bones. So are you saying that all rolls in bvh/fbx are lost and twist bones not used ? In particular we have to decompose xz to bend bones and y to twist bones when retargeting, see #1709 how I retarget G2 to G8 for example, with twist bones.

    Unless I misunderstand what you mean.

  23. Thomas Larsson repo owner

    No, ,that is not what I meant. What matters is not the roll angle, but the twist relative to the rest pose. If the bone is untwisted, the X and Z axes are oriented as in rest pose, which differs from the orientation in another rig with different roll angles.

    The T-pose at frame zero is used to calibrate this difference. E.g., if we want to retarget from T-pose with zero roll angles to A-pose with zero roll angles, we cannot just copy the pose in world space, because that would introduce twisting.

    https://diffeomorphic.blogspot.com/p/bvh-how-it-works.html

  24. Alessandro Padovani

    I can confirm that “center animation“ makes the blender bvh the same as the script. I still don’t understand why they should be different without “center“, I mean a walking figure on a path should be the same, if we “center” then we lose the path, and this may not be desirable.

    Below an example extracted from another issue where we may not want to “center”. Here the root bone is the same but the animation is not, as in the picture above, unless we “center” both losing the path.

    ‌

  26. Konstantin Konstantinov

    I have an idea for such corrupted rigs like “videoplayback__UE_Face.fbx“ with a lot of extra leaves. What if we just remove those leaf bones before any retarget attempt? All we need is to find the leaf bones with no weighted verts in the mesh, and remove them. In this model only third (distal) finger limbs have weighted verts in the mesh, no bones after them have any vertex groups. The same for eyes and toes.

    Of course, the rig is an anomaly, it’s not something which must be supported. The “_end[_end]..“ bone chains are the user’s fault. So it’s rather not for the add-on, but for users, encountering such problematic ones.

    Btw, have anybody tried to orient the last in chain bones (not leaves) with mesh verts locations? What if we use the averaged vector from the bone head to each vertex (like the bvh importer does when averaging children heads to find the parent direction)? Does it worth trying?

    ‌

  27. Alessandro Padovani

    You can’t rely on weight maps because animations may include only the armature, and usually they do. Anyway I understand Thomas already prunes the extra leaves in some way, probably by bone name comparing with the standard UE armature. Or they are just ignored for retargeting.

    p.s. The problem with the test file by Bouich is not that there’s extra leaves, but that they are messed up and positioned in the wrong way, thus making index_03 and the other fingers at the wrong orientation. This is usually not the case and a “bug“ in the specific file, that Thomas is trying to work around anyway.

    My guess is that the animation was first exported without extra leaves, thus making the last bones z-up as expected since fbx only gets joints. Then it was re-exported with extra leaves, but at this time the last bones were already in the wrong orientation.

  28. Konstantin Konstantinov

    Yes, in a bvh there is no mesh, so it’s all guesswork. :-( But if we have such a big clue in an fbx, it’s a sin not to try. Of course, it’s not for retargeter, but rather a separate tool to fix problematic rigs using any possible clues, kind of ‘last resort' or ‘if nothing helps’ tool. I’d try to play with it this way, and then show here where it comes.

  29. Thomas Larsson repo owner

    After lots of attempts I think the test file comes in rather nicely now.

    FBX bones that end with the string “_end” are removed. This is mostly cosmetic, since those bones are not considered in the retargeting process. Before the finger tips, i.e. the last real leaf bone, were reoriented in the same way as its parent (i.e. the second finger link). However, for this to work the F-curves would need to change too, so I discarded that idea.

    Instead the finger tips are not included in the reference T-pose anymore, and neither are the thumb bones. This seems to work much better. The plugin records the difference in rest pose for these bones, and compensates for that during retargeting. This is the same thing that happens for the spine. Spine bones can have very different orientations (in particular, the hip may point in the wrong direction), but a spine at rest should always be mapped to a spine at rest.

  30. Alessandro Padovani

    Then I guess you could do the same to fix the issues with collar bones, that is, do not include them in the reference T-pose, same as the spine.

    p.s. However, I stress again that the test file by Bouich is faulted to start with, usually there’s not these issues in well exported files. So whatever you did to fix this issue I’d keep it as an option, as “try to fix bad leaf bones“ or something.

  33. Log in to comment
Assignee
–
Type
bug
Priority
minor
Status
resolved
Votes
0
Watchers
3
Jira: the preferred issue tracker for Bitbucket. Join the team!