develop version of daz importer crashes blender on import

[Jens Hoffmann]

While playing around with the latest develop version of daz importer, I found that blender (v3.1.2) reproducibly crashes on import of specific characters. (Just “Import DAZ” without any additional processing is enough for the crash.)

The same DAZ characters import with the released importer v1.6.1 and blender v3.1.2 without problems.

After a lot of experimenting I found out that the crash is related to the geograft “New Genitalia for Victoria 8”,

specifically to the node “New Genitalia For Victoria 8 - Color Layer”. If I delete the color layer node in DAZ before export, the import will no longer crash blender.

(However, when exporting just the base genesis 8 figure with NGV8 including color layer works, but with figure adjustments and clothes, the color layer becomes a no go.)

From the backtrace, my guess would be that the color layer node creates some faulty mesh where blender trips over when applying the subdiv modifier to the geograft.

If you need additional info or want me to test something specific, just let me know.

‌

console output:

Reloading external rigs...
Reloading external metarigs...
Loading BVH Retargeter
BVH Retargeter loaded

Loading MHX RTS
MHX RTS loaded

Loading DAZ
DAZ loaded
Load settings from C:/Users/Jens/import-daz-settings-28x.json

Loading D:\Data\DAZ\User\Export\Character\Test\Test.duf
Parsing data
Creating reference to target figure:
<BoneInst lThighTwist-5 N: lThigh F: None T: <Figure /export/character/test/test.duf#FMJeans 1 None> P: None R:None>
Creating reference to target figure:
<BoneInst lMetatarsals-2 N: lFoot F: None T: <Figure /export/character/test/test.duf#FMJeans 1 None> P: None R:None>
Creating reference to target figure:
<BoneInst rThighTwist-5 N: rThigh F: None T: <Figure /export/character/test/test.duf#FMJeans 1 None> P: None R:None>
Creating reference to target figure:
<BoneInst rMetatarsals-2 N: rFoot F: None T: <Figure /export/character/test/test.duf#FMJeans 1 None> P: None R:None>
Preprocessing...
Fitting objects with dbz file...
Zero verts: New Genitalia For Victoria 8 - Effect Layer
Zero verts: Geometry Shell 3
Building objects...
TIFFFetchNormalTag: Warning, Incompatible type for "RichTIFFIPTC"; tag ignored.
Dependency loop: lToe lMetatarsals
Dependency loop: rToe rMetatarsals
Pruning UV maps
Scale eye moisture vertices for Genesis8-female mesh "Test Mesh"
Centers: <Vector (0.0305, -0.0629, 1.4779)> <Vector (-0.0285, -0.0628, 1.4776)>
File "D:\Data\DAZ\User\Export\Character\Test\Test.duf" loaded in 13.533 seconds
Import scene only
Cycles Settings:
transparent_max_bounces: 8 < 16
volume_bounces: 0 < 4
Eevee Settings:
shadow_cube_size: 512 < 1024
shadow_cascade_size: 1024 < 2048
use_shadow_high_bitdepth: False != True
light_threshold: 0.009999999776482582 > 0.001
sss_samples: 7 < 16
sss_jitter_threshold: 0.30000001192092896 < 0.5
use_ssr: False != True
use_ssr_refraction: False != True
use_ssr_halfres: True != False
ssr_thickness: 0.20000000298023224 > 0.02
ssr_quality: 0.25 < 1.0
ssr_max_roughness: 0.5 < 1.0
Render Settings:
hair_type: STRAND != STRIP
Light "Light" settings:
shadow_buffer_clip_start: 0.05000000074505806 > 0.01
shadow_buffer_bias: 1.0 > 0.01
use_contact_shadow: False != True
contact_shadow_bias: 0.029999999329447746 > 0.01
contact_shadow_distance: 0.20000000298023224 > 0.01
contact_shadow_thickness: 0.20000000298023224 > 0.1
Cycles Settings have been updated to minimal requirements for this scene.
Eevee Settings have been updated to minimal requirements for this scene.
Render Settings have been updated to minimal requirements for this scene.
Light "Light" settings are insufficient to render this scene correctly.
See http://diffeomorphic.blogspot.com/2020/04/render-settings.html for details.
Error : EXCEPTION_ACCESS_VIOLATION
Address : 0x00007FF6B25C3254
Module : blender.exe
Thread : 00001b34
Writing: C:\Users\Jens\AppData\Local\Temp\blender.crash.txt

backtrace

Exception Record:

ExceptionCode : EXCEPTION_ACCESS_VIOLATION
Exception Address : 0x00007FF6B25C3254
Exception Module : blender.exe
Exception Flags : 0x00000000
Exception Parameters : 0x2
Parameters[0] : 0x0000000000000000
Parameters[1] : 0x000001CA635DE5D0

Stack trace:
blender.exe :0x00007FF6B25C30F0 draw_subdiv_build_cache
blender.exe :0x00007FF6B25C3BC0 draw_subdiv_create_requested_buffers
blender.exe :0x00007FF6B25C4A60 DRW_create_subdivision
blender.exe :0x00007FF6B25F10F0 DRW_mesh_batch_cache_create_requested
blender.exe :0x00007FF6B25E7E60 drw_batch_cache_generate_requested
blender.exe :0x00007FF6B25C1D80 drw_engines_cache_populate
blender.exe :0x00007FF6B25BE4D0 DRW_draw_render_loop_ex
blender.exe :0x00007FF6B25BF5A0 DRW_draw_view
blender.exe :0x00007FF6B2F27820 view3d_main_region_draw
blender.exe :0x00007FF6B28CD940 ED_region_do_draw
blender.exe :0x00007FF6B24C60C0 wm_draw_window_offscreen
blender.exe :0x00007FF6B24C5F10 wm_draw_window
blender.exe :0x00007FF6B24C5A10 wm_draw_update
blender.exe :0x00007FF6B24A0110 WM_main
blender.exe :0x00007FF6B21003C0 main
blender.exe :0x00007FF6B77CFE74 __scrt_common_main_seh
KERNEL32.DLL :0x00007FFCA94954D0 BaseThreadInitThunk
ntdll.dll :0x00007FFCAACA4830 RtlUserThreadStart

‌

[Thomas Larsson]

I don’t experience any crash, but I get an error message about an illegal diffuse texture, That message was introduced recently in commit b1223aa (version 1.6.1.940) because of a problem that led to a crash. If you have an earlier version I suggest that you update.

Pruning UV maps
Illegal diffuse texture in new_gens_V8_1840_Anus-2:
<bpy_struct, ShaderNodeTexImage("?V
*") at 0x000000002A0A5608>
Scale eye moisture vertices for Genesis8-female mesh "Genesis 8 Female Mesh"
Centers: <Vector (0.0329, -0.0639, 1.6862)> <Vector (-0.0329, -0.0639, 1.6862)>
File "D:\home\bugs\geografts\g8f-ngv8.duf" loaded in 1.868 seconds

‌

[Jens Hoffmann]

The develop version I tested with was commit b96226c from yesterday.

I’ll let you know if I find out any more about the exact condition for the crash.

And just in case that matters: the version of NGV8 I used is from 22.02.2021 (with support for genesis 8.1.)

I also have the NGV8 Expansion Pack from 01.11.2021 installed.

[Jens Hoffmann]

To verify that the issue is really with the subdiv and NGV8, I modified geometry.py at line 167:

   if (self.type == "subdivision_surface" and
        self.data and
        (self.data.SubDIALevel > 0 or self.data.SubDRenderLevel > 0)):
        if (ob.name == "New Genitalia For Victoria 8"):
            print("skipped NGV8")
            subDLevels = 0
        else:
            print('create subdiv for %s' % ob.name)
            mod = ob.modifiers.new("Subsurf", 'SUBSURF')
            mod.render_levels = subDLevels
            mod.levels = self.data.SubDIALevel
            if hasattr(mod, "use_limit_surface"):
                mod.use_limit_surface = False
            self.data.creaseEdges(context, ob)
    else:
        subDLevels = 0

so that no subdiv is build for NGV8. And with that blender no longer crashes on “DAZ Import”.

Strangely though, I can now add the subdiv modifier by hand and blender still has no issue.

The geometry also looks fine in the geometry editor.

EDIT:

I could fix the crash by simply moving the subdiv modifier below the SubD Displacement modifier:

    if (self.type == "subdivision_surface" and
        self.data and
        (self.data.SubDIALevel > 0 or self.data.SubDRenderLevel > 0)):
        subDLevels = self.data.SubDIALevel + self.data.SubDRenderLevel
    else:
        subDLevels = 0

    for dmat in self.materials.values():
        level = dmat.getSubDLevel(0)
        if level > subDLevels:
            mod2 = ob.modifiers.new("SubD Displacement", 'SUBSURF')
            mod2.subdivision_type = 'SIMPLE'
            mod2.render_levels = level - subDLevels
            mod2.levels = 0

    if (subDLevels > 0):
        mod = ob.modifiers.new("Subsurf", 'SUBSURF')
        mod.render_levels = subDLevels = self.data.SubDIALevel + self.data.SubDRenderLevel
        mod.levels = self.data.SubDIALevel
        if hasattr(mod, "use_limit_surface"):
            mod.use_limit_surface = False
        self.data.creaseEdges(context, ob)

Now, I don’t know, if this makes any sense since I do not know the displacement modifier (I’m a blender newbie) but it fixes the blender crash never the less.

So even if the reordering makes no sense, it highlites that the crash is related to having both modifiers applied.

[Alessandro Padovani]

daz studio 4.15.0.30, blender 3.1.2, diffeomorphic 1.6.2.0953

I don’t get any issue either. Tried with g8f + ngv8 + pear morph + basic wear + toulouse hair. Test scene included ngv8.duf. You may upload a test scene yourself as simple as possible to reproduce the issue for us to check.

steps:

1. import the test scene ngv8.duf

‌

[Alessandro Padovani]

test scene for ngv8

[Jens Hoffmann]

Hi Alessandro,

I do get the crash with your example!

Which raises the question what is different in my installation that triggers the crash. Which version of NGV8 do you use and do you have the NGV8 expansion pack? Also: what modifiers do you have on the ngv8 mesh after plain import?

EDIT:

I loaded your example in DAZ, saved it under a new name, exported the dbz and loaded it with “Import DAZ” in blender.

I have: daz studio 4.20.0.2, blender 3.1.2, diffeomorphic 1.6.2.0953

[Jens Hoffmann]

To get back to the modifier issue, I see now that "SubD Displacement" is just another renamed subsurf modifier. Why would you add multiple subsurf modifier on the same mesh (without any other modifier in between?) Maybe I miss something but that seems strange to me.

[Alessandro Padovani]

The test scene works fine for me.

I don’t have the expansion pack so I don’t use it in the test scene. Can’t figure out what settings could trigger the issue but below are mine. I use the standard export not HD, works both with standard and easy import. NGV8 update 2017.09.09.

‌

[Jens Hoffmann]

Yeah, here is the difference. I tried your settings and now I also get no crash.

After checking each option individually, the culprit seems to be “Auto Smooth”. With it disabled, I get the crash and when enabled no crash.

What happens if you uncheck “Auto Smooth”?

EDIT:

guess I found the reason why this makes a difference:

With the “Auto Smooth” option enabled, blender now shows the hint “Autosmooth or custom normals detected, disabling GPU subdivision” in the subsuf modifiers.

So it’s probably just some bug in blender 3.1.2 related to GPU subsurf with multiple subdiv modifier.

Note: in importer v1.6.1 only one subdiv modifier was created for the mesh. So the issue was not triggered there.

EDIT 2:

Just to sum it up: With the option “Auto Smooth” enabled, the import will not crash.

The created Mesh for NGV8 will have the 3 modifiers:

Armature "SkinBinding"
SubDiv "Subsurf"
SubDiv "SubD Displacement"

but the SubDiv will use software rendering instead of GPU.

If I now go to the NGV8 Mesh / vertex data → Normals, the moment I uncheck “Auto Smooth”, blender will crash. But if I remove the second SubDiv modifier before uncheking auto smooth, blender won’t crash. So blender 3.1.2 seems to have an issue with one armature modifier and two subdiv modifiers using the GPU on the same mesh.

EDIT 3:

seems to be a known bug in blender (https://developer.blender.org/T96283)

Note: The described condition is the case here: multiple subdiv using GPU and last one set to level 0. So, this issue can be closed.

But still, does it actually improve something to have multiple subdiv modifier vs one subdiv with higher level?

[Alessandro Padovani]

Nope, works fine here with and without “auto smooth“. Tried both cuda and optix don’t get any error. May be it depends also on the graphics card and drivers ? I see the bug report you’re pointing to is for linux.

I have windows 10 and a gtx 1060 with 472.39 nvidia driver.

The second subd is a simple subdivision so it doesn’t use smooth and it’s much faster. Is used to tessellate the geometry for the displacement map, if the figure use it. You can also use adaptive subdivision instead but we opted for a second subd because adaptive is experimental yet.

See #924.

[Jens Hoffmann]

May be it depends also on the graphics card and drivers ? I see the bug report you’re pointing to is for linux.

Yes, might be driver related. In any case, I can reproduce the crash with the example blend of the blender bug ticket. (windows 11, nvidia rtx 3080, driver 512.15)

The second subd is a simple subdivision so it doesn’t use smooth and it’s much faster. Is used to tessellate the geometry for the displacement map, if the figure use it.

I can see why you would add the simple subdiv, if a material has displacement and there was no subdiv otherwise, but still I think a single subdiv (with the max required level adjusted) per mesh should be enough.

But I think the code is not optimal anyway, since it loops over all materials and creates an additional subdiv, if the materials needs one and has a higher render level. But the subdiv is applied to the mesh not the surface. So if your mesh would have 10 materials with displacement and increasing level, then you would add 10 additional simple subdiv on the mesh. That's probably not what was intended.

Maybe something like this would work. (untested)

    if (self.type == "subdivision_surface" and
        self.data and
        (self.data.SubDIALevel > 0 or self.data.SubDRenderLevel > 0)):
        mod = ob.modifiers.new("Subsurf", 'SUBSURF')
        mod.render_levels = subDLevels = self.data.SubDIALevel + self.data.SubDRenderLevel
        mod.levels = self.data.SubDIALevel
        if hasattr(mod, "use_limit_surface"):
            mod.use_limit_surface = False
        self.data.creaseEdges(context, ob)
    else:
        mod = None
        subDLevels = 0

    for dmat in self.materials.values():
        level = dmat.getSubDLevel(0)
        if level > subDLevels:
            if not mod:
                mod = ob.modifiers.new("SubD Displacement", 'SUBSURF')
                mod.subdivision_type = 'SIMPLE'
                mod.levels = 0
            mod.render_levels = subDLevels = level

Did a simple test render with cycles, and with the NGV8 I do not see any obvious differences. But that mesh is probably not a good test for displacements.

[Thomas Larsson]

The code has been changed so only a single subsurf modifier is created, with the max number of render levels. However, I didn’t test the case with simple subdivision because I cannot find the relevant test file.

[Alessandro Padovani]

It’s ngv8.duf attached here. Works fine for me.

[Alessandro Padovani]

As for commit 446c496, if I check the ngv8 mesh I see it doesn’t add the simple subdivision anymore, but a single catmull subdivision at the highest level required.

Be aware that this adds a heavy workload that’s not necessary. Again simple subdivision is much faster that’s why it’s usually used for displacement instead of catmull. That is, they both exist for a reason and provide different tools. Catmull is to smooth organic forms. Simple subdivision is for fast tesselation.

Please go back to simple subdivision or at least provide an option. Having catmull at the highest level is killing the performances.

‌

[Jens Hoffmann]

I made a comparison with a reasonably big mesh and one catmull subdiv with render level 3 vs a catmul subdiv with render level 2 + a simple subdiv with render level 1. And yes, the version with two subdivs is a bit faster. The difference was not huge but meassureable in render times.

So now, I agree with Alessandro that it makes sense to keep them separate. (But maintain the change to create only one simple subdiv instead of possibly multiple..)

    if (self.type == "subdivision_surface" and
        self.data and
        (self.data.SubDIALevel > 0 or self.data.SubDRenderLevel > 0)):
        mod = ob.modifiers.new("Subsurf", 'SUBSURF')
        mod.render_levels = subDLevels = self.data.SubDIALevel + self.data.SubDRenderLevel
        mod.levels = self.data.SubDIALevel
        if hasattr(mod, "use_limit_surface"):
            mod.use_limit_surface = False
        self.data.creaseEdges(context, ob)
    else:
        subDLevels = 0

    displacementSubDLevels = max([dmat.getSubDLevel(0) for dmat in self.materials.values()])
    if displacementSubDLevels > subDLevels:
        mod = ob.modifiers.new("SubD Displacement", 'SUBSURF')
        mod.subdivision_type = 'SIMPLE'
        mod.levels = 0
        mod.render_levels = displacementSubDLevels - subDLevels

It might make sense to add an option to set the viewport level of the simple subdiv to something else than 0 though, to work around the blender bug that leads to the crash. Like an option “Show displacement subdiv in viewport” and then set mod.levels to 1 instead of 0.

‌

[Alessandro Padovani]

imho

Displacement is only visible at render time since eevee doesn’t support it. Thus setting the viewport simple subd other than zero is not useful and it slows down things. The “bug“ is only for some cards/drivers and must be resolved on the blender side. I mean if we start to patch diffeomorphic for all the temporary blender bugs then it’ll become a mess soon.

What you can do is to turn off “auto smooth“ until the blender guys fix it.

[Thomas Larsson]

OK, the second subdiv modifier is reintroduced in the last commit.

[Alessandro Padovani]

Commit e358b5d works fine thank you for the fix.

[Jens Hoffmann]

Displacement is only visible at render time since eevee doesn’t support it. Thus setting the viewport simple subd other than zero is not useful and it slows down things.

I disagree here. Yes you are right what it makes no sense if all you want ist to render in cycles, but for people who want to export the model, it can definately make sense.

Most exporters only consider the viewport settings and ignore render settings. (i.e. FBX export with “Apply Modifiers” only evaluates viewport settings.) So if you want to export with them evaluated, you would need to go over all meshes by hand and enable them for export. Why not simplify that task?

[Alessandro Padovani]

Because usually subdivision is exported as subdivision, not baked. Even with fbx. You can’t export jcms if you bake the subdivision. Then most game engines may have their own way to handle tessellation for displacement maps. It may be dynamic for example. This is why it makes little sense to export the tessellation for displacement anyway.