Better Material Handling with Attributes

[Midnight Arrow]

I’ve been doing a lot of coding on my own Python material importer lately, and I’m considering taking another crack at replicating the Uber shader, which I’d be happy to share here. But as I’m sure you’re aware, in order to get around the volume “fix” in Blender 3.3, materials need to be converted to UDIMs and collapsed into a single material, which obviously means getting rid of the imported material assignment from Daz Studio. The material slot system is brittle in general (#2115), and since Daz Studio and Blender use very different ways for handling materials, this makes it hard to convert between the two programs. Matching names is prone to fail due to the fact that material names in Blender must be unique while DUF material files rely on standardized naming.

However, I have come up with a much better way to handle imported materials.

The solution is to ignore the material slots entirely.

The attribute system that geometry nodes uses can store completely arbitrary data per element, and one of those datatypes is a string. During import, all you need to do is copy the Daz Studio material name as an attribute onto the imported mesh’s face.

With Python, this is trivial.

attribute:Attribute = mesh.attributes.new("diffeo_original_material", 'STRING', 'FACE')

for index, polygon in enumerate(mesh.polygons):
  mi:int = polygon.material_index
  attribute.data[index].value = mesh.materials[mi].name

It’s equally easy to fetch the indices of the polygons assigned to that material so they can be operated on.

def get_face_indices_by_original_material(mesh:Mesh, material_name:str) -> []:

  attribute:Attribute = mesh.attributes["diffeo_original_material"]
  face_indices:[] = []

  for index, element in enumerate(attribute.data):
    if element.value == material_name:
      face_indices.append(index)

  return face_indices

I’ve successfully implemented this system for my one-click material converter, and it works extremely well.

By storing Daz Studio’s material assignment per-face, the mesh’s material slots can be deleted and reconstructed as many times as you want, while the original data remains intact. This would make it much easier to code a DUF material importer (which I am also thinking about doing). To future-proof the Diffeomorphic importer and take advantage of improvements in Blender, I recommend moving away from dealing with material slots and instead preserving the imported data as attributes, so that slots can be treated as output of operations on the cached attributes_,_ rather than data itself.

Attributes can also be used to store arbitrary data for shader nodes. For instance, the bump strength of each material can be stored as a float attribute and plugged into the bump node in the shader tree, eliminating the need to have a bump strength slider on the combo material. It’s also very easy to save attributes to disk as JSON files and load them, so they can be shared between models (of the same generation).

I plan to recreate the Uber shader as a single material with attribute masks and UDIMs, so implementing this would be a prerequisite for me contributing my work on that here.

[Alessandro Padovani]

I don’t get you. Materials has nothing to do with slots. We can have as many materials as we want in the scene, linked or not it doesn’t matter. To keep unlinked materials it is enough to set a fake user we don’t need nodes. As well we can have as many slots as we want without any linked material.

A daz studio material gets a unique name in blender by suffixes as .001 if the daz name is duplicated, because as you say materials in blender are global. Then a geometry slot can link any material in the scene of course.

‌

[Midnight Arrow]

That’s something completely different.

If you delete a slot, the faces lose what material they were assigned to because the material indices are wiped.

My proposal is to store the material name (as it was assigned in Daz Studio) on a per-face basis, so it isn’t lost when slots are deleted. Then even if you delete all the slots, you can import a DUF material file and it will create new materials and assign them based on the material name stored on the geometry.

Or, to use the case of #2115, when the Genesis 9 figure is imported every face is assigned the Daz Studio-specific material name (“Body”, “Arms”, “Face”, etc.). So even if the material slots are corrupted, you can run an operator to delete all the slots and repopulate them with blank materials based on the attribute data.

[Alessandro Padovani]

That doesn’t sound good to me, especially for HD objects. Then Thomas may have a different opinion.

p.s. I also don’t get what issue this would fix, #2115 works fine, of course we don’t have to merge materials if we need the full list.

[Midnight Arrow]

Currently, material indices are stored as integer attributes right now anyway. This proposal is what is happening under Blender’s hood. If file size for HD models is a concern, you could just duplicate the already-existing integer attribute and store the names of the Daz materials as an EnumProperty on the mesh, and index into that. In fact that might be better, because you could also use the EnumProperty to select materials in the UI panel.

The Blender Foundation have already converted data like UVs, edge creases, material indices, etc. into attributes because they’re much more robust than hardcoded data inside a C struct. This proposal, to pivot towards using attributes to store an arbitrary and unlimited amount of data, brings Diffeomorphic in line with the future of Blender.

Edit:

The point of this isn’t “don’t merge materials if you need the full list”, the point is “the data is stored no matter what, so we can recall it anytime we need it”. Even if you do merge materials, you can fetch the data you need as a safety measure.

[Midnight Arrow]

I did some tests on a Genesis 8 at Subdiv level 3, with over one million polygons.

Storing a copy of the material indices inside an 8-bit integer attribute took up about one megabyte (versus over 100Mb for the figure). In the unlikely event a figure is imported with more than 256 materials, then it’ll need to use a 32-bit integer (4Mb) instead.

Performing a loop over the faces to extract the indices took less than a second from the Python console.

The upside is that this proposal caches and preserves imported data DUF files depend on. For instance, the Import DAZ Material operator relies on material slots. If they’re changed, it doesn’t work. However, if that operator was altered to operate on a cached attribute instead, it could remove all the material slots, rebuild the correct ones, and assign the faces to the correct material indices as needed. This value outweighs the downside of a relatively minor increase in memory usage. Besides, the importer has a Finalize panel, so “utility attributes” like these can be cleaned up if the user doesn’t want them anymore.

[Alessandro Padovani]

That’s why there’s an option to use material names instead of slots, in general the material operators can use names. Slots can be ignored. Again, I fail to see what benefits or fixes this would bring, other than using more memory and being slow. Then again, Thomas may have a different opinion or see things for the code that I don’t.

[Thomas Larsson]

Thank you for this info. I haven’t worked with attributes and it will take some time before I get comfortable with them, but it is good to know about this possibility.

[Midnight Arrow]

Attributes are just vertex groups on steroids.

A vertex group maps a float value to a single vertex, but attributes can map any data type to any element. Boolean to face, integer to edge, color to vertex, etc. You can use them basically the same way, to store any kind of data you want.