Strange Shapkey Artifacts in Unity When Exporting

[Hovhannes Kirishchyan]

I am trying to import some fbx models with shapekeys to Unity.
I have read the article on main website “Unwinding Actions“ but I don’t think that it is the problem as I am using facial animations that are breaking the model not on middle values of shapekeys but literally every value makes it worse.

The artifact basically completely destroys the geometry of the mesh as soon as I modify the shapekeys.

I have tried this with multiple options on HD non_HD with and without subdivision modifiers. in all cases the model is broken.

Example:
When using the standard Viseme Shapekey “AA” Only the mouth should open. But instead the geometry on the whole model breaks, this is especially the case with years and eyes.

Unfortunately this really breaks the model and spoils the game that I am trying to make.

NO Shapekeys:

AA Shapekey to 1, the artifact is noticeable around EARS and EYES:

Played with some Shapekeys to make the artifact especially noticeable:

This gets especially notable when I have the textures on.

The body also gets totally destroyed.

The first image is with no shapekeys maxed and the other one is with all of them maxed.

As far as I understand the shape keys of the face should not affect the vertices of the body, but this happens all of a sudden.

If anyone has a solution please help I have spend a lot of time trying to fix it. Thank you.

‌

‌

[Xin]

You can’t port a so called “control rig” as the one generated by this addon (or rigify, or mhx) to a game engine just like that, the rig would be completely broken if exported without proper care (as seems to be your case here). This is particularly true in this case since there are a lot of drivers, constraints involved, on top of a complex bone hierarchy.

Game engines impose a lot of limitations on the type of rigs they can make use of, the type of constraints they can handle, the bones' hierarchy they can deal with, etc.

That’s why you need to perform an entire middle step that partially depends on the engine, as explained here (I recommend you watch this series fully, it will help you a lot with porting to game engines): https://www.youtube.com/watch?v=2GQwAM5Wodk&list=PLdcL5aF8ZcJvCyqWeCBYVGKbQgrQngen3&index=1

The basic idea is that you create a simple, naked rig, stripped off any complex relations and then bake actions from the control rig to this simple rig (through copy transforms constraints). Then you export that simple rig. Again, this requires you to be careful to prepare the “game rig” properly, as the videos above explain.

Doing this is beyond the scope of this addon, which just gives you a proper control rig within Blender so you can animate daz figures easily, within Blender. What you do afterwards is up to you. Trying to go beyond that would make this addon massive and extremely complex (entire addons exist just to try to convert rigify rigs to game rigs).

[Hovhannes Kirishchyan]

Dear Xin,

Earlier I have had this issue with my model and was able to fix it. The “middle step“ that you have mentioned is to my understanding included in the addon. I just don’t remember how I’ve managed to work around this problem. Shape key in it’s nature in just a data on linear interpolation of vertices from one point to another. I shouldn’t encounter a problem like that at all when importing to Unity.

Regardless, thank you for your time I am sure the videos that you have provided me with will be helpful in some other cases.

Best,

H.K.

[Xin]

The face sliders which daz uses and which this addon creates in Blender don’t dial just shape keys, but a combination of shape keys and bones' poses. Those bones' poses are the ones that are problematic to port to a game engine.

If you just want a pure shape key (no bone poses at all), then you can use Blender’s tools to generate it:

1. Create a copy of the base mesh and remove its armature modifier and shape keys (or just import the base mesh again and remove the armature).
2. Dial whatever combination of face sliders you want to get the expression you want in the original mesh.
3. Click the original mesh, then shift+click the duplicated mesh and go the shape keys panel and click “Join as Shapes”. A new pure shape key will be generated that corresponds to the full deformation of the original mesh.
4. Put the original mesh in rest pose again. To transfer this shape key back to the original mesh, click the duplicated mesh, then shift+click the original mesh and in the shape keys panel click “Transfer Shape Key”.

Now the original mesh has a pure shape key that doesn’t depend on any bones.

The problem with using a lot of shape keys on game engines, especially on meshes with a lot of vertices, is that performance starts to take a hit (also they are quite large in terms of memory).

Also, the rig will still be broken due to the problems I mentioned before, so you still want to learn how to export the rig properly to Unity, so you can later export skeletal animations to it.

[Thomas Larsson]

It looks rather like vertex numbers have been corrupted. Do the shapekeys look good in Blender?

[Alessandro Padovani]

I second what Xin reported. This is just to add that for game engines you may find it easier to use G1 G2 figures since they don’t use jcms and have a simpler rig with no twist bones, other than using “pure” shapekeys.

Then as a trick to optimize the memory you could separate the head and the body, this way face shapekeys will take only the head geometry.

[Hovhannes Kirishchyan]

Dear Thomas Larsson,

Yes they do look normal, I don’t see the strange things happening in Blender.

[Hovhannes Kirishchyan]

Dear Xin,
After doing exactly as you said, the geometry of the model is still broken.
I have duplicated the mesh of my model, set the shape keys on the original one and then transferred them to the new mesh with no bones, then I applied the new mesh as a shape key to the original mesh.

The problem seems to be not in bone poses. I think the Convert to Shape keys button creates pure shape keys on it’s own.

This way or another the problem is not solved.

[Xin]

Try to export a mesh with only a single pure shape key and no rig to see if that works.

[Hovhannes Kirishchyan]

I have done just like that on the screenshot above, a single shape key.

[Alessandro Padovani]

it may also be a scale issue since morphs needs to be scaled if you use a global unit other than 0.01, see #645

[engetudouiti]

It is not scale issue from seeing the pics, but it seems normal issue for me at least about pure shape key case.

Did you try to set normal for blend shape options in unity?

Though it is modo topic, but I think it related with your issue. (if it is pure blender FBX exporter of your used blender version, the best thing is report it for blender dev as FBX exporter and importer problem)

https://community.foundry.com/discuss/post/1168615/content/114194

usually import <> export need to check both aprication setting (and importer exporter)

[Alessandro Padovani]

Yes of course vertex normals have to be recomputed when animating, including shapekeys. So you should have an option in unity for that. But this is not a blender or diffeomorphic issue, it’s unity. Please note that if you have a static object with custom normals, as it may happen with props and environments, this does not apply.

From the link by Engetudouiti. “Try in Unity .. set both Normals and Blend Shape Normals to "Calculate" in the model section of the fbx“.

[Hovhannes Kirishchyan]

Dear Alessandro Padovani, Xin and Thomas Larsson thank you for your time, you were really helpful. I am happy to announce that after nearly a month I finally found the solution to the problem.

To those interested, If you are exporting the model for a video game this should be helpful for both Unreal and Unity.

1 Problem:

The strange artifact around the ears of the character was due to the fact that when exporting the model blender recalculates normals for each of the shape keys. To avoid this you should choose to export FBX with smoothing across faces or if you want more precision across edges.

2 Problem:

Shape key artifacts around the body mesh are due to the problem within Unity. When importing a large-sized model Unity uses smoothing of the mesh relative to normals, since the normal of the mesh are from a low poly version and are using smoothing within blender itself smoothing is not imported and you get smoothing of a low high-poly mesh by normals of a low-poly one that creates the problem.

Solution:

After setting the smoothing across the faces or edges of the model within Blender when exporting FBX, go to Unity and set the import setting to Import the Blend Shape Normals instead of Calculating them.

Big thanks to everyone! Xin the videos were very cool I learned a lot.

‌

P.S.

This fixes the problem yet it would be better to do it all within blender so that the normals will match the Geometry. But this is absolutely optional since importing normals for shape keys is also a good solution.

[Hovhannes Kirishchyan]

After setting the smoothing across the faces or edges of the model within Blender when exporting FBX, go to Unity and set the import setting to Import the Blend Shape Normals instead of Calculating them.

[Alessandro Padovani]

Or simply change the smoothing angle from 60 to 95 in your unity import settings.