Questions about reading .duf character files without .dbz

[Noel Lang]

I was reading the 'Export To Blender' documentation and am curious about the hurdles that necessitated the use of the .dbz method for diffeomorphic.

‌

From what I gather, Daz clothing fitting projects the figure's shape morphs onto the clothes. The Tafi Unity store - the official creators of Daz Studio - the G8 figures come with a clothing auto-fit functionality within Unity. Here, the Unity code simply projects the clothes onto the body mesh. Is there any additional complexity to this process within Daz Studio itself?

‌

Instead of trying to reproduce the final vertex and bone locations inside Blender, these basic data are exported by a custom script to a .dbz file, which is then used in Blender to recreate the final meshes.

I'm not entirely sure what is meant by the "final bone locations" in this context.

From what I understand, DAZ projects a bone morph, which can be perceived as a 'bend'. This is nothing more than a remote control for adjusting the xyz rotation of the bone in a certain manner. I would assume you could look up exactly how these are affecting rotation and reverse engineer the bone positions from the A pose reference?

‌

Thank you for all your hard work, Thomas! Absolute legend

[Alessandro Padovani]

As I understand it, dbz allows to bake all the morphs you use to make the figure in daz studio, aka actor, so you only import the morphs you need for animation, aka pose controls.

I’m not sure what you are “proposing“ here.

[Noel Lang]

I am not proposing anything, simply asking questions. I am trying to find out the challenges to importing the .duf file without starting Daz Studio for my own program. Before trying it myself, I’d like to know what I’m in for.

[Alessandro Padovani]

Well of course it’s possible to import as unmorphed then use ERCs, but this is not how blender works and greatly degrades performances, similar to what happens in daz studio which always loads all the morphs references. Of course you’re welcome to contribute if you find bugs or have suggestions for improvements. Or if you want to share some code.

If there’s nothing to add I’d close as invalid since this isn't a bug report nor a proposal.

[Noel Lang]

Well I’m wanting to bake the morphs within Blender, I’m aware the way Daz deals with it is not how Blender works. What I’m working on is well outside the scope of this addon, and I’m looking to understand why the dbz method was chosen instead of baking directly in Blender.

Feel free to close as invalid if there’s no further information.

[Thomas Larsson]

You could also try to import the scene with mesh fitting set to Morphed (characters), and then perhaps transfer shapekeys to clothes. However, I have never really used that option, and it doesn’t seem to work fine. In particular, ERC morphs seem to be ignored even if they are turned on in the global settings.

However, the final bone locations are stored in the duf file iirc, so maybe the importer can get the bone right without ERC morphs. I will check.

[Thomas Larsson]

The main reason for dbz is that shapekeys have to be transferred to clothes and geografts, and I didn’t have a good way of doing that in Blender at the time.

[Noel Lang]

Ahhh, I see. Thank you very much! That’s great to hear

Yea, Morphed (characters) doesn’t really work, I was worried there might have been a monumental roadblock the dbz file was overcoming.

[Alessandro Padovani]

Well of course if we can avoid dbz and bake in blender that would be awesome, since it cuts down the export time to zero, especially for HD. But that also means we have to import all the actor morphs including ERCs for baking, that will slow down the import time. Overall may be dbz is easier.

p.s. Then there are other things requiring dbz, as sbh and dforce hair that are baked from the daz viewport, as well as mesh smoothing for outfits. Not to mention that without dbz the HD shape is to be taken from dhdm files as Xin does.

[Thomas Larsson]

Actually, morphed import seems to place the bones correctly. Below is the same file, imported with unmorphed, morphed and dbz fitting. But sometimes some shapekeys are missing for some reason, which means that the mesh is wrong and doesn’t fit the correct armature. Will investigate.

‌

[Thomas Larsson]

Now all baked morphed should be loaded, i.e. the same that are listed as baked morphs when you import dbz.

[Alessandro Padovani]

Marking as resolved since improvements are implemented, though it wasn't the original purpose of the discussion.

[Noel Lang]

Just got the chance to try this out now.

‌

Just tried Victoria 8, seems like the bones are placed correctly as you say, but shapekeys are missing.

‌

‌

[Noel Lang]

Some shapekeys still don't import with morphed import

[Thomas Larsson]

Could you provide an example? I’m importing a character that is 30% Victoria and 70% Aiko and the shapekeys are there.

[Noel Lang]

Sure, here are two example figures (I had ERC morphs and baked morphs enabled).

testcharacter1.duf - Genesis 8 Female

Morphs:

• Height (/data/Daz 3D/Genesis 8/Female/Morphs/DAZ 3D/Body/FBMHeight.dsf) - -59.59%%
• Head Propagating Scale (/data/DAZ 3D/Genesis 8/Female/Morphs/DAZ 3D/Base/SCLPropagatingHead.dsf) - 15%
• 2 Morph Loader Pro imported morphs from OBJ

‌

And then a second test figure with Morph Loader Pro morphs removed in case those are not possible to import without dbz.

testcharacter2.duf - Genesis 8 Female
Morphs:

• Height (/data/Daz 3D/Genesis 8/Female/Morphs/DAZ 3D/Body/FBMHeight.dsf) - -100%
• Head Propagating Scale (/data/DAZ 3D/Genesis 8/Female/Morphs/DAZ 3D/Base/SCLPropagatingHead.dsf) - 25%

Left is morphed, right is DBZ imported

‌

‌

[Noel Lang]

Hmm, I wasn’t able to figure out how to upload them directly in here. So here is a Google Drive link to the test characters instead: https://drive.google.com/drive/folders/1xDBAHCZB7TH_U2EEoFCw1I7asKleDaQH?usp=sharing

[Noel Lang]

A bit of a related question. If I have a scene file with a character + environment, is there any harm in using 'Morphed'? Will the results of importing the environment be the same as if it were on Unmorphed?

[Thomas Larsson]

Sorry, I forgot about this issue. The Tifa Lockhart morphs are now imported, but the height morph is not. But when I make a fresh character with the height morph, it is imported. It turns out that the plugin creates two assets for the G8F mesh when loading your file, but only one with mine. The URLs differ in case:

"/data/DAZ%203D/Genesis%208/Female/Genesis8Female.dsf#geometry"
"/data/Daz%203D/Genesis%208/Female/Genesis8Female.dsf#geometry"

The solution is to make the asset table case insensitive. I will see if I can do that without breaking everything else.

‌

[Noel Lang]

Awesome. Is it possible for morph loader pro morphs to be supported as well?

[Thomas Larsson]

Now the test character is imported correctly. There were actually two issues here:

1. The file part of URLs are now case insensitive, but the local part (after the #) are case sensitive.
2. If the morph value is negative, the shapekey min value must be set before the shapekey value.

What is morph loader pro?

‌

[Noel Lang]

Ah, I forgot I already gave you a morph loader pro .duf with testcharacter1.duf. Morph loader pro morphs seem to load perfectly, thank you!

‌

testcharacter2.duf (shared in the same google drive link) doesn’t appear to import correctly. The Head Propagating Scale morph doesn’t load and the character is below the floor. See here, left is Morphed, right is DBZ.

‌

I’ve tested a few characters, quite a lot work, which is fantastic. However, the ones that had missing morphs were:

• Kanade 8
• CO Reine (which produced an error, see below)

Loading D:\Personal\DAZ3D\KnowlangContent\People\Genesis 8 Female\Characters\CO Reine.duf
Parsing data
Preprocessing...
Building objects...
BLI_dynstr_append text too long or format error.
Traceback (most recent call last):
  File "D:\Other Games\Blender\PROD\scripts\addons\import_daz\error.py", line 222, in execute
    self.run(context)
  File "D:\Other Games\Blender\PROD\scripts\addons\import_daz\main.py", line 225, in run
    self.loadDazFile(filepath, context)
  File "D:\Other Games\Blender\PROD\scripts\addons\import_daz\main.py", line 169, in loadDazFile
    asset.build(context, inst)      # Builds morphs 1
  File "D:\Other Games\Blender\PROD\scripts\addons\import_daz\modifier.py", line 760, in build
    Formula.build(self, context, inst)
  File "D:\Other Games\Blender\PROD\scripts\addons\import_daz\formula.py", line 64, in build
    asset.build(context, inst, value)
  File "D:\Other Games\Blender\PROD\scripts\addons\import_daz\modifier.py", line 760, in build
    Formula.build(self, context, inst)
  File "D:\Other Games\Blender\PROD\scripts\addons\import_daz\formula.py", line 64, in build
    asset.build(context, inst, value)
  File "D:\Other Games\Blender\PROD\scripts\addons\import_daz\modifier.py", line 760, in build
    Formula.build(self, context, inst)
  File "D:\Other Games\Blender\PROD\scripts\addons\import_daz\formula.py", line 64, in build

Repeats the last bit for a long time then:

  File "D:\Other Games\Blender\PROD\scripts\addons\import_daz\formula.py", line 57, in build
    ref,key,value = self.computeFormula(formula)
  File "D:\Other Games\Blender\PROD\scripts\addons\import_daz\formula.py", line 103, in computeFormula
    asset = self.getAsset(ref)
  File "D:\Other Games\Blender\PROD\scripts\addons\import_daz\asset.py", line 55, in getAsset
    ref = getRef(id, self.fileref)
  File "D:\Other Games\Blender\PROD\scripts\addons\import_daz\asset.py", line 409, in getRef
    id = normalizeRef(id)
  File "D:\Other Games\Blender\PROD\scripts\addons\import_daz\asset.py", line 417, in normalizeRef
    ref = quote(id)
  File "D:\Applications\Blender\3.6\3.6\python\lib\urllib\parse.py", line 881, in quote
    return quote_from_bytes(string, safe)
  File "D:\Applications\Blender\3.6\3.6\python\lib\urllib\parse.py", line 905, in quote_from_bytes
    if not isinstance(bs, (bytes, bytearray)):
RecursionError: maximum recursion depth exceeded in instancecheck

‌

[Thomas Larsson]

The morphed option imports morphs that are shapekeys, but it ignores formulas for the bones. This is a limitation, at least for now. To some extent formulas are baked into the rest pose, because the bones are located at their preview position, but this does not affect the mesh. What is missing in the second test character (actually also in the first) is the SCL Propagating Head morph, which scales the head bone and its children. You can manually load that as a custom morph.

[Thomas Larsson]

The infinite recursion should be gone now.

[Noel Lang]

Daz Studio 4.21.05 | Blender 3.6.1 | Diffeomorphic 1.7.2.1798 | Windows 10

Infinite recursion is fixed, thanks!

but it ignores formulas for the bones

Ah, ok. I see.

CO Reine and Kanade 8 still don’t load head and body morphs.

Left is morphed, right is dbz.

‌

[Thomas Larsson]

I checked Kanade, and the problem is that she also uses a formula, CTRLKanade which uses the shapekeys FBMKanade and FHMKanade. Since this is the normal way characters are made, not being able to import such files makes the morphed option rather useless.

But this is a problem that has already been solved. If we manually import CTRLKanade as a custom morph, the shapekeys are created. Instead of trying to reinvent the wheel, I am trying to reuse the code that imports morphs in general. The idea is that instead of importing the morphs when they are found, the importer should just collect them, and import them at the end using the same code as manual morph import. Work in progress.

[Noel Lang]

That’s fantastic.

I’ve checked CO Reine, and she also uses a formula, CTRLCOReine.dsf which uses FHMCOReine and FBMCOReine shapekeys.

[Thomas Larsson]

The new way of morphed import is implemented now, more or less. The baked morphs are listed in the morphs panel if the global Baked Morphs option is enabled:

‌

However, there are a number of known issues, to be dealt with later:

1. Only one morph per file is imported. Despite what picture says above, there is only one Tifa Lockhart shapekey. This happens because I reused the code that imports morphs, and there is normally one morph in each file. But in this file there are three morphs, and the two last are missing.
2. Morphs may be misassigned if the scene several figures of the same type (e.g. two G8F’s).
3. Shapekey transfer to clothes gives bad result, because the shapekeys should be zeroed before transfer.

‌

[Thomas Larsson]

The transfer issue should be resolved now.

[Thomas Larsson]

Now the morphs are assigned to the correct instance when there are several instances of the same figure. And all morphs are found if there are several in the same file. The import custom morphs tool also imports multiple morphs; the standard morph tools don’t because there isn’t any standard morph file with multiple morphs, I think.

[Noel Lang]

Daz Studio 4.21.05 | Blender 3.6.1 | Diffeomorphic 1.7.2.1808 | Windows 10

Awesome, just did a retest on Kanade 8 and CO Reine. Not sure if I’m doing something wrong, but it doesn’t seem to import correctly yet.

‌

Kanade 8, morphed (right), dbz (left). Only body morph loads, and the rig is misaligned.

Baked morphs (morphed)

‌

Baked morphs (dbz)

‌

[Noel Lang]

Daz Studio 4.21.05 | Blender 3.6.1 | Diffeomorphic 1.7.2.1808 | Windows 10

CO Reine morphed (left), dbz (right). Morph looks like base G8F.

Baked morphs (morphed)

‌

Baked morphs (dbz)

‌

[Thomas Larsson]

I also noticed the issue with Kanade. The problem is that the ctrl file only contains a reference to the fbm file but not to the fhm file. I don’t understand why it works in DS.

I don’t have Reine, but can you check if the shapekeys have been loaded and that the value = 1. Ifthere is a problem with the drivers the shapekey values are = 0.

The Victoria and Aiko morphs are loaded, but Aiko misses a global scale = 95%. The scale is not in the duf file so it must be somewhere in the definition of the morphs. The plugin misses the scale also if the morph is imported as a custom morph.

‌

[Thomas Larsson]

Aiko loads correctly now, including the 95% scale. This still causes problems when merging rigs and transferring shapekeys.

[Alessandro Padovani]

My personal opinion on this.

As I see it, morphed is not useful. I mean if we want to bake the figure and work “the blender way“ then we import as dbz. If we want to use the base figure with morphs and work “the daz way“ then we import as unmorphed then import morphs, eventually as ERCs. Unless I miss something, since I only use “the blender way“.

[Thomas Larsson]

I don’t really use the morphed option either, but if it exists it should at least work properly. Plus working on it led to some improvements, e.g. that object scale are included when loading morphs and that merge rigs and transfer shapekeys work with non-zero morphs.

[Noel Lang]

Morphed would be a big plus for me, since it’d allow me to import without having to launch Daz.

A few use cases I’ve come across:

• I work in a team, all our assets are on a shared drive. I’m the only one who uses Daz, so if we want to import a character, I have to launch Daz, import and send it off to them. With a functioning morphed option, they can import characters themselves (and they do, for all other assets).
• Sometimes I’m using Blender on Linux or another Windows machine without Daz, I still have access to the network drive, so it’d be a time saver not to have to switch machines/OS.
• Final one would just make it easier for my addon that simplifies importing assets (including through diffeomorphic).

Yea, specific use cases and I don’t really expect these to be supported, but it’d definitely be a nice to have.