UnicodeDecodeError on import of a basic scene

[Kat]

Daz: 4.20.0.17, Blender: 3.4.1, Diffeo: 606318a, OS: Linux

In attempting to recreate another, completely different issue (that I so far haven’t been able to track down the cause of), I found this UnicodeDecodeError on a relatively basic scene with just a base Genesis 8.1 Female and Toulouse Hair. This issue occurs on both regular and easy import, using the DBZ import and BSDF materials (though it seems to occur on extended principled, too).

‌

Regular import error:

Python: Traceback (most recent call last):
  File "/home/xyz/.config/blender/3.4/scripts/addons/Diffeomorphic-import_daz-606318a11d0d/error.py", line 214, in execute
    self.run(context)
  File "/home/xyz/.config/blender/3.4/scripts/addons/Diffeomorphic-import_daz-606318a11d0d/main.py", line 229, in run
    self.loadDazFile(filepath, context)
  File "/home/xyz/.config/blender/3.4/scripts/addons/Diffeomorphic-import_daz-606318a11d0d/main.py", line 182, in loadDazFile
    asset.postbuild()
  File "/home/xyz/.config/blender/3.4/scripts/addons/Diffeomorphic-import_daz-606318a11d0d/cycles.py", line 121, in postbuild
    self.tree.postbuild()
  File "/home/xyz/.config/blender/3.4/scripts/addons/Diffeomorphic-import_daz-606318a11d0d/cycles.py", line 1977, in postbuild
    hasDiffuse = self.diffuse and marked.get(self.diffuse.name)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb7 in position 1: invalid start byte

Easy import error:

Python: Traceback (most recent call last):
  File "/home/xyz/.config/blender/3.4/scripts/addons/Diffeomorphic-import_daz-606318a11d0d/error.py", line 214, in execute
    self.run(context)
  File "/home/xyz/.config/blender/3.4/scripts/addons/Diffeomorphic-import_daz-606318a11d0d/main.py", line 729, in run
    self.easyImport(context)
  File "/home/xyz/.config/blender/3.4/scripts/addons/Diffeomorphic-import_daz-606318a11d0d/main.py", line 739, in easyImport
    bpy.ops.daz.import_daz(
  File "/home/xyz/Downloads/blender-3.4.1-linux-x64/3.4/scripts/modules/bpy/ops.py", line 113, in __call__
    ret = _op_call(self.idname_py(), None, kw)
RuntimeError: Error: Python: Traceback (most recent call last):
  File "/home/xyz/.config/blender/3.4/scripts/addons/Diffeomorphic-import_daz-606318a11d0d/error.py", line 214, in execute
    self.run(context)
  File "/home/xyz/.config/blender/3.4/scripts/addons/Diffeomorphic-import_daz-606318a11d0d/main.py", line 229, in run
    self.loadDazFile(filepath, context)
  File "/home/xyz/.config/blender/3.4/scripts/addons/Diffeomorphic-import_daz-606318a11d0d/main.py", line 182, in loadDazFile
    asset.postbuild()
  File "/home/xyz/.config/blender/3.4/scripts/addons/Diffeomorphic-import_daz-606318a11d0d/cycles.py", line 121, in postbuild
    self.tree.postbuild()
  File "/home/xyz/.config/blender/3.4/scripts/addons/Diffeomorphic-import_daz-606318a11d0d/cycles.py", line 1977, in postbuild
    hasDiffuse = self.diffuse and marked.get(self.diffuse.name)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfa in position 0: invalid start byte
Location: /home/xyz/Downloads/blender-3.4.1-linux-x64/3.4/scripts/modules/bpy/ops.py:113

‌

[Alessandro Padovani]

It may help to know your regional settings, if you’re on a english system or otherwise. As for utf8 there’s with and without bom, it looks like python is expecting bom ? In my system that’s a win10 with english regional settings the duf files are utf8 without bom and I don’t get any error so this seems specific to linux.

Here your duf test file opens as utf8 without bom with no errors.

[Kat]

Sorry for the delay in responding. This has been an incredibly elusive issue, but after an identical issue was brought to my attention by another person (also on Linux), I looked a bit more into it.

Now using Diffeo 25965f7 (I don’t believe any changes made between the two commits would have affected this issue), I’ve found that when the Material Method is set to BSDF, the UTF-8 issue from before occurs, however it does not occur with the mode being set to Extended or Single Principled. Instead, for those, I get an “unsupported shader” warning as shown below.

‌

Is it possible that this file is related to the error? While it would be convenient, I am also inclined to believe otherwise because the exact byte that the UTF-8 error displays is different between any two otherwise identical import attempts.

Finally, both Morphed and DBZ mesh fitting modes behave identically, so the dbz file/it’s data (or lackthereof) is unlikely to be the cause.

[Alessandro Padovani]

The “unsupported shader“ refers to the daz shader and should be independent from the material method so it’s strange you don’t get the warning for bsdf. Anyway it’s only a warning meaning that you should convert to uber before exporting. Please note that iray does the same it converts to uber in background.

In daz select all materials and apply the iray uber shader. It’s one click and will convert all to uber for blender.

[Kat]

After digging around some more, using a stripped down version of the issue demonstration duf above, I have discovered that this issue is in this specific example related to the Genesis 8.1 Female Cornea material slot.

When the cornea’s Refraction Weight is set to 1.0, Top Coat Weight is any nonzero value, and the cornea has a normal map applied, the error occurs, however, when there is seemingly any other configuration, it is okay.

Here are some examples in which the error does and does not occur.

• Refraction Weight: 1.0, Top Coat: 1.0, w/ Normal : Error (This is the default material configuration for the Genesis 8.1 Female model.)
• Refraction Weight: 1.0, Top Coat: 0.5, w/ Normal : Error
• Refraction Weight: 0.99, Top Coat: 1.0, w/ Normal : No Error
• Refraction Weight: 1.0, Top Coat: 0.0, w/ Normal : No Error
• Refraction Weight: 1.0, Top Coat: 1.0, w/o Normal : No Error

‌

[Kat]

Following a short discussion with the acquaintence mentioned before, who has also been having this issue, I am inclined to believe that this issue is a general issue in the material tree generation with Diffeo. While I have tracked this issue to one specific material setting in that example file, I don’t believe that is a universal answer and that on a more complicated model or scene, it may manifest in completely different configurations.

For instance, he has issues with Theo Brows under both the asset default configuration and a completely stripped down plain white material. It seems to be less so the brows themselves than something that the brows do to the wider scene or model, since the issues persist even after stripping out that particular asset.

[Thomas Larsson]

I think I know what happened. The plugin first builds a node tree, and at the end deletes nodes that are not connected to the material output. Thus the diffuse node is deleted if refraction weight = 1.0, because all nodes that come before refraction don’t contribute to the material. But the plugin kept a reference to the removed diffuse node which was probably overwritten by something else.

I think the issue is fixed in the last commit. If it isn’t, you could disable the Prune Node Tree global setting, and prune the node tree manually with the button in the Advanced Materials panel.

[Alessandro Padovani]

@Kat If there’s nothing to add I’ll mark as resolved.

[Kat]

Sorry again for the delay. After a quick test with the example duf, I have confirmed this issue is resolved, at least for this specific case.