BVH retarget too much upper bones incorrect positions
Issue #2179
resolved
Hi,
I noticed this bug with literally almost everything, i am using a VMD here in attachment but literally everything
I made a video here
it make all dance the animation really different than the original, ( the upperarm always quickly flash move)
This is really a big issue, because almost any animation we import have this issue with the upper arm.
Thank you!
Comments (23)
-
-
- changed status to open
-
repo owner
That the retargeter has problems with shoulders is a known issue, that I don’t have a good solution to. Some time ago I introduced a tool to clear the rotation of selected bones, keeping the world space orientation of its children unchanged. The main reason is to quickly fix the collar bones if the shoulders are sloping, cf https://diffeomorphic.blogspot.com/2024/03/fixing-sloping-shoulders.html.
However, it does not work in the present case, where we want the collar bones to tilt upwards. A quick way to change the pose is to change the pose at one range, and the Shift Animation of selected bones.
-
What about the “relax shoulders“ idea, do you think it may be feasible ?
-
reporter
- attached Mixamo.fbx
- attached Unreal.fbx
Hi Thomas and Alessandro,
This issue happens also with mixamo and UE rig!!
I tried both, same issue.
Please find in attachment, the same animation in Mixamo and in UE rig,
Please note the UE rig ( fingers are NOT properly tracked its not important i will open a different issue another day)
However for test purpose please find mixamo and ue rig you will notice the same " flash" of the shoulders.
Thank you,
-
reporter
The shift / clear bones doesnt seem to fix this issue, because this is different than Diffeomorphic: Fixing Sloping Shoulders i believe.
Here its the upper arm not the clavicle that is causing the issue.
-
Yes of course it happens with mixamo and unreal too, what did you expect ? The issue is in the animation itself, not in the rig. Again, the shoulder bones are not used thus the mesh breaks on a realistic human figure. It doesn’t break with “robot“ figures because they have a rigid torso with no shoulder movements.
Again, the retargeting is correct, it is the animation itself which doesn’t fit human figures. What Thomas could try is the “relax shoulders“ idea to make the shoulders follow, but don’t know if it is practical to implement or doable at all.
-
importer 4.2.1.2286, retarget 4.2.1.0052, blender 3.6 and 4.2
bug for Thomas.
I tried to retarget the mixamo animation and got a crash. I did a manual install of the latest addons available.
steps:
- import G8
- load and retarget “mixamo.fbx“
Deleting temporary FBX objects Traceback (most recent call last): File "C:\Users\Alessandro\AppData\Roaming\Blender Foundation\Blender\4.2\scripts\addons\retarget_bvh\utils.py", line 418, in execute self.run(context) File "C:\Users\Alessandro\AppData\Roaming\Blender Foundation\Blender\4.2\scripts\addons\retarget_bvh\retarget.py", line 611, in run info = self.retarget(context, filepath) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Alessandro\AppData\Roaming\Blender Foundation\Blender\4.2\scripts\addons\retarget_bvh\retarget.py", line 630, in retarget srcRig = self.readMocapFile(context, filepath) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Alessandro\AppData\Roaming\Blender Foundation\Blender\4.2\scripts\addons\retarget_bvh\load.py", line 220, in readMocapFile deleteObjects(context, list(imported_objects)) File "C:\Users\Alessandro\AppData\Roaming\Blender Foundation\Blender\4.2\scripts\addons\retarget_bvh\load.py", line 648, in deleteObjects bpy.ops.object.select_all(action='DESELECT') File "C:\Program Files\blender-4.2.1-windows-x64\4.2\scripts\modules\bpy\ops.py", line 109, in __call__ ret = _op_call(self.idname_py(), kw) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: Operator bpy.ops.object.select_all.poll() failed, context is incorrect
-
repo owner
I’m unable to reproduce Alessandro’s bug. It should be gone in the latest commit, but perhaps the fbx objects are not properly deleted.
The flipping problem is related to gimbal lock. An example is the flip between frames 165 and 166 if the fbx animation is loaded to G9:
The eulers for the selected bone jump:
If we enable Quaternions in the global setting when we import G9, the rotation of the upper arm changes smoothly. However, the animation still jumps in the viewport.
-
repo owner
The reason why quaternions don’t help for G9 is that the long upper-arm bones are not deform bones. Instead, deformation is done by the twist1 and twist2 bones in the tweak collection, which are driven by the corresponding (drv) bones in the hidden collection.
The twist bones are driven by an Euler Y rotation, and is thus subject to gimbal lock even if the driving bone isn’t.
Unfortunately converting the rig to mhx doesn’t help either, because the G9 constraints are still present. Mhx for G8 should work (haven’t checked), because then there are no drivers inherited from the daz rig, and the plugin can set up more stable drivers itself (damped track + copy rotation on all axes).
-
reporter
Thank you Thomas, i did a quick test with the last build i can confirm it worked great for the G8! you were 100% right.
However for the G9 no, issue still there.
Thank you again!
-
reporter
I tried a little trick like to export the baked animation from the G8
then i import it to the G9 with the convert to G8 on , but it didnt work even like this, the issue appeared.
-
Commit adaf9c9 works fine, thank you for the fix.
solution. better quaternions. I was going to do the same but was blocked by the crash, so wasn’t able to test on G8 G9 and was fooled by the original rigs, sorry for that. As for the gimbal lock this can be easily avoided by using quaternion on the twist bones. Unfortunately fixing the bones isn’t enough as the jcms still rely on euler so don’t work fine, we have to set the jcm drivers to quaternion as well.
That is, to avoid flipping we use quaternion in the rig for upperarm and thigh, because they are free joints on all axes, but this is not enough we have to use quaternion also for their driven bones and drivers. This is the same for any rig where we use quaternions, including G1 to G9.
steps:
- import G9 with jcms
- set the drv twist bones to quaternion
- change the cbs_upperarm drivers to quaternion
- import the animation, the flipping is fixed and the shoulders work fine
p.s. In general we could use all quaternion bones in a rig, and some blender rigs do that. But in practice euler is good enough if the joint is not free on all axes, and euler is better readable, so we use quaternions only for free joints.
-
repo owner
The G9 twist bones now use quaternions, both for the bone and driver rotation modes. This is not quite right, because there should be a driver for the W channel as well, but at least the jumps are gone.
-
reporter
hmmm tried last build, issue seems to still be there with the G9.
frame 687 for example.
-
Commit 9426348 works fine here.
If you are referring to frame 575 and some others, I’m afraid this is expected and works fine, or as fine as it can. If you try the same pose in daz studio you will get the same result. This is because bones and jcms work far outside of the daz limits, and because as noted before the animation is unnatural and doesn’t properly use the shoulder bones. The fact that we fixed the euler flipping doesn’t fix the bad animation and I’m afraid there’s nothing we can do about it.
Than again Thomas may have better ideas, but for me this is as good as it gets.
-
reporter
I dont understand, it work great for G8 but not G9
still same issue please look :
there is huge difference and the jumps are still massively there on the G9 but the G8 is perfect.
i have the last commit.
-
reporter
Ok i have figured it out, i had to activate Quaternions in the global settings.
-
reporter
I am sorry the results in G9 still looks different than G8 and terrible.
here is a video that explain everything:
-
repo owner
The previous commit was not good. I misunderstood how to drive the Y channel, and missed to drive the W channel to make the quaternion normalized. The last commit addresses both issues, and I think the result looks quite reasonable now. Here are the drivers for the r_upperarmtwist1 bone:
-
reporter
WOW thank you so much!! now i can confirm it’s great!!!
if nothing else can be closed, thank you again!
-
Yes, 575 is improved thank you I didn’t think about the W axis at all. Closing as resolved then.
-
- changed status to resolved
- Log in to comment
I’m afraid this is just the animation itself that’s too bad. If we load the animation on the standard Miku Hatsune model in Miku Miku Dance, or in Blender using the MMD Tools, we see the arm rotation breaks the mesh. This is because the rotation relies only on the arm bone, without using the shoulder bone, which is not realistic and would break any real human arm. Thus the retargeting does the same, correctly.
Then Thomas may try to “beautify“ bad animations, but personally I have no idea how. Or you can try to fix the animation yourself by tweaking the shoulder bones to a more realistic pose. This is my own opinion though, Thomas may have better ideas.
p.s. If this happens often in mmd animations as you say, then it means mmd animations in general are not suited for realistic human models, and they need to be fixed by hand. Indeed mmd uses toon models where the issue is there but less visible because of the low resolution of the models.
p.p.s. relax shoulders. @Thomas, one idea could be to have a “relax shoulders“ option to split the arm rotation between the shoulder and arm bones, eventually with a percentage. For example 30% relax means the shoulder takes 30% of the arm rotation. This may help to fix bad animations where shoulder bones are not used, but I have no idea if this is practical to implement or doable at all, or how good the result would be as it may take the arms out of place. Let us know.