Material import overrides

[xmf]

I think the materials imported by daz importer are often incorrect. The disparity also widens when you consider eevee and cycles differences. In my opinion extended principled is almost useless since it introduces a lot of botched materials especially in the context of eevee for example if you are importing a scene you are better of using single principled you will get much more cleaner results. But even single principled suffers from crazy bump, specular, roughness and clearcoat values. A piece of clothing is often imported with roughness higher then 1, which is simply incorrect. When it comes to bump the strength value is often above 1, which in my opinion is incorrect. A bump strength value should be between 0 and 1 because it represents the blend values above 1 will have inconsistent results based on distance to the camera. It is not always noticeable but the principle is there, how much the bump is visible should be controlled through the bump distance and strength should be either 0 or 1 with proper distance if your bump map is a proper height map and not some albedo with grayscale.

I understand this is not a simple issue and you cant please everyone. But I think there could be a more leeway when it comes to import. You could add a way to override materials with a custom material based on names, or add some rule system you could offer a list of values that will be rewritten if they exceed certain values so if roughness value on pbsdf is 1.1 it will re-interpret it and make it 0.75 which is more correct.

[xmf]

I think a system for custom materials could be very simple if you could figure out a way to uniquely name each material, perhaps a hash sum of all their parameters? Then you could add two options, one specifying a path to blend file containing custom materials and one specifying a path to a text file with a list of materials to be replaced. So if you have a material named Hair-9A6CCF75F3 in blend file and in that text file you have Hair-9A6CCF75F3 in the list then whenever you import that specific material it will automatically replace it so you could eventually build a library of material setups that just work with blender out of the box and best thing is these blend files and text files can be shared if daz assets are not embedded in the blend file.

[xmf]

Also the text file could specify the original daz library path, so you could easily fix all the texture-paths of all replaced materials. So I guess this would assume on the both sides you use only one daz library, im not sure how and if you would reliably manage the variabilities of multiple libraries on either or both sides.

[xmf]

Well I guess you can always just run fix missing textures in blender. But here is a better idea, instead of manually creating a text file you can add a button to shader pane to export for daz importer usage so it will automatically export all the data it needs into a json so it can reliably import and replace that material.

EDIT: Actually i think you may even not need to export it to an external file, just save it into the blend file and then when importing get the text file directly from the materials blend.

[Alessandro Padovani]

You can already do that with material palettes. Import your figure then customize the materials as you wish and save the palette. You can then reuse the palette with any figure having the same material names. See #947 unfortunately this is not documented yet.

[xmf]

This is not the same as what I am suggesting. Material names are not going to be unique and you cant easily create a library of materials that can be shared universally with pallets. Its just copy pasting materials onto a plane, its nothing you cant do with default blender. I am not sure if it does anything other then copy pasting materials around, if not then i think its a waste of button space its like it is there to let you know you can copy paste materials around.

What I am suggesting, if it could work, is much more in-depth. People could easily create large libraries of material conversion and share them and it would work universally for everyone and you would not need to copy paste anything around. For example you could have a library of all DAZ Original figures converted to a custom setup and you could include this in diffeomorphic addon itself so everyone can use it with click of a button. Or you could have a place where people share their libraries, so if someone converts 1000 figures and it all looks good/better then the automatic diffeo conversion and they share this file anyone can easily make use of it.

EDIT: Also the point of my suggestion is to shorten the DAZ to Blender pipeline. Pallets hardly do anything for that. Because the more steps and gimmicks you have in between, the less likely is for people to use it. Materials issues is probably the largest stretch of that pipeline. Even if its not a problem for you to convert materials, just the extra step that you have to do every single time you want something new is very bothersome and not to mention that you have to organize everything you did so far on your own. Its been made easier with new asset browser in blender, but still it feels like a third wheel for something it should be 2.

[Alessandro Padovani]

I don’t see the difference, you can make your custom palettes for G9 and share them with other users for example. Then applying the palette is just one click away.

[xmf]

Here let me make it more obvious for you:

• Palettes are essentially just a another way to copy paste materials in blender.
• My suggestion is to create a streamlined and intelligent way to manage daz-related materials within the context of diffeomorphic importer.

‌

• Pallets require manual copy pasting. They don't transfer file paths. They are a step that you do additionally after importing. There is no intelligent organization of material names unless you manually rename them all and then maintain it manually.
• My suggestion would require only setting a path to the material blend library. The file paths would be intelligently transferred. It does not require additional steps, meaning once you enable material overrides they will automatically be used instead of default diffeomorphic materials meaning you just import it and it works out of the box.

‌

The way diffeomorphic importer imports materials right now, as i understand it, is by interpreting the daz material and then though an algorithm recreate it in blender. This is a task that is impossible to automate correctly, it is comparable to trying to synthesize speech with conventional programming. The only real solution for diffeomorphic daz importer to ever have a real usable out of the box materials is to manually create a database of materials and you wont do that yourself that is why you need a community to get involved if you want to get it done and i think it will be done if there is a way to do it.

So yeah - difference is a diffeomorphic importer with useless materials that 99% of the time need manual fixing and diffeomorphic importer that works 100% of the time with materials/assets that either you or the community fixed. Again, palettes is just copy pasting materials, it was there before diffeomorphic importer and it doesn’t really do anything tangible to actually improve diffeomorphic material imports. Its a completely different thing actually.

‌

So yeah, if you still dont understand please be more specific. I think my suggestion is relatively simple improvement code wise, but can potentially big ripple effect in improving diffeomorphic usability.

‌

[Alessandro Padovani]

Your idea doesn’t work in my opinion because for custom palettes anyone can do his own “better“ version, so you would need to maintain the material library anyway. Pasting palettes is more flexible and easy.

[xmf]

I think you are missing the point of my suggestion. The point is that anyone can make a “better” version, the end user can then choose the version they want if its shared publicly and it will work without any gimmicks. There is no need for maintenance of materials library because the names of materials would be unique to begin with, only thing you would need to do is just adjust the material and save the library and thats it anyone can use it.

Imagine having a scene with 200 materials or a clothing set with 20-30 materials. What now you want to select each and every one of those objects and manually replace materials on every one of them. Its just not going to happen. You need to somehow automate this in intelligent way, nobody should be wasting life on copy pasting materials from planes when it can be done in a more sane way.

[xmf]

Additionally it could be possible to have multiple libraries and create a priority order. So you can download 10 libraries and if some materials overlap then the one in higher priority gets used.

[Alessandro Padovani]

I understand your idea but I still believe it doesn’t end well to maintain such a huge material library as you’re proposing. But the plugin is scriptable if you want to try.

https://diffeomorphic.blogspot.com/p/daz-importer-operators-version-162.html

[xmf]

I disagree with you especially since you don't even bother to elaborate your conclusions and the most interesting thing you can contribute to the thread is “i dont like it” and “go make it yourself”. Its just a suggestion and I already made workarounds for myself but this is not suitable for public, my suggestion is a general improvement for benefit of everyone. So whatever.

But as I understand Thomas is the main developer so if he bothered to read through the thread I would love to hear his opinion as to why this is or isn't a good idea.

[Alessandro Padovani]

Yes of course that’s my own opinion, Thomas may very well decide otherwise or help if you want to script it.

[xmf]

Yeah if you call your trolling an opinion then you are correct - that is your opinion.

[Thomas Larsson]

Hm. I don’t quite understand what you are saying. It sounds like you suggest a huge database with manually created Blender materials for every asset. That is not going to work, few people will be willing to do unpaid work in the long run. And what about quality control, who is going to throw out low-quality work from a community database? Sorry, I was once on the MakeHuman team and have seen such initiatives fizzle out.

I think that the only general-purpose solution is to use the information available in the daz materials. It of course not straightforward to translate iray into cycles or eevee, but that is the information that is available in general. Perhaps one could invent some custom pipeline for a few select assets, like the Genesis skins.

[xmf]

Actually I am not suggesting a huge database with manually created Blender materials. What I am suggesting is to create a functionality to make that possible and see what happens. It is not about doing unpaid work and quality control doesnt matter because nobody is going to use something that sucks so if a library is shared on one of the .blend sharing sites you can easily see the popularity of the librart there and if someone is serious enough about making a decent package they can just show pics in descriptions or make a youtube video about it. Again the suggestion is not for daz importer to manage the community or to create libraries, the suggestion is to create a core functionality so that people can do it themselves. I am not trying to create a huge burden for you with my suggestion, i mean who would want to manage something so massive? Well, you wouldnt and I probably wouldnt but if there is foundation for it then someone might be interested to do it and then everyone would benefit but even if not it would still be beneficial its just that library distribution would not be centralized

Fixing materials is work but its not huge amount of work and if we could share that work in very streamlined way i think it would boost the feasibility of DAZ to Blender pipeline.

I understand its a big thing however you look at it, but considering there is no other way to get “proper” materials when importing fom DAZ i think its at least worth thinking about it.

[Thomas Larsson]

Perhaps such functionality already exists. All objects that have been imported with the plugin has a DazUrl property, which contains the url to the asset. This property is displayed in the Active Object field of Utilities panel, cf. https://diffeomorphic.blogspot.com/p/utilities-panel-version-16.html. Since it is a unique identifier (up to case) it could be used as a key to a material database.

So this could be the workflow:

1. Import the character with converted materials.
2. Use the DazUrl property to search for better materials in some database.

‌

[xmf]

But then you are basing your key to the material on an object? I think that is not very useful. Someone can apply material set from one figure onto another and then on the import you will get the incorrect match. You need a way to uniquely identify materials. Do you think that is possible somehow? Maybe if you take all the parameters that make up the material and create a hash and use that as a key? But this would need to be built into daz exporter.

[Alessandro Padovani]

Perhaps one way could be to save a blender material palette in the daz library together with the asset. If the palette has the same name as the asset then the materials in the palette will be loaded with the asset. The user interested in creating a blender material library for daz assets can do the usual daz distribution file in zip format that will contain the palettes and folders for the addressed assets, to be unzipped as usual in the daz library.

This basically provides a sort of “automatic loading“ for palettes #947.

[Thomas Larsson]

I think identifying materials globally is impossible, because materials can be tweaked by the user and are saved in the local scene file. So materials basically only have local urls. To define a character, you start with a base character, say G8F, and add morphs and materials. The mesh topology, uv sets, and bone hierachy are universal and are referred to with url links, and the plugin exposes those urls (perhaps not for uv sets). The materials and combination of morph values are character specific.

There is already a database for materials for iray and 3dlight; it is called the daz store. I suppose that vendors could sell blender materials through the same channel. However, providing blender materials using other people’s textures doesn’t seem like a good idea, since it may lead to legal problems.

[xmf]

I don't understand your logic. If material is tweaked then it is no longer the same material hence the different hash. The whole point is to provide a replacement material for specific import so that people can just load a library called for example “Daz Originals” and you can then use diffeomorphic importer as usual and have decent materials without need to fiddle around with palettes or anything else just load the library and use it and thats it. If its that simple then people will create and share large libraries, but if there are so many obstacles like there are now then majority of those that would otherwise bother - wont. Palettes are terrible for managing anything other then few materials especially because it affects only single object so sure a skin pallete i get it but clothing pieces? Scenes?

If you want to port Victoria 8 figure it assumes you will not tweak the materials in daz, so the hash will always be the same. The benefit of my suggestion is that the material library would work even if Victoria 8 materials are applied for any other figure which is not the case if you are basing you library key on the figue/object. And I never suggested distributing textures or daz assets, you can adjust the file paths on import so they match but you need to keep some data around to be able to do that as far as i am aware this is perfectly legal no assets are being redistributed.

[Thomas Larsson]

Sorry, but I just don’t get it. AFAIK the only way to store Blender materials are as blend files; that is how Blender materials are distributed e.g. on Blendswap. A blend file without textures makes no sense to me.

[xmf]

The blend file does not store textures into the blend file unless you explicitly store them. Otherwise its just nodes with with file paths, so the person that creates the blend material library will have paths for their textures in their daz library and all you have to do then when importing is just fix those paths for all the materials and it will work. But you need that fixing step. Diffeomorphic importer already has daz library paths so you have everything you need to fix the texture paths during the import process. There is no need to store textures.

‌

[Thomas Larsson]

OK, now I think I understand what you are saying. You need a tool that loops over all materials of a mesh. If some texture is missing, the path is replaced with something that starts with one of the daz root path, if such a file exists. That would be doable, and then you could distribute materials with palettes as Alessandro suggested.

A database has to be indexed by object urls, though. Unlike the situation for mesh topology or bone hierarchy, the duf files contain the full material definitions rather than pointers to other files. so the importer cannot know about material urls.

[xmf]

Maybe it would be an improvement to the palettes but it has nothing to do with my suggestion.

You don’t need material url’s. If the material definition you can access in a duf file is unique(if material is unique) then you can run a python hash() function on that definition and it will give you a unique string representing that material and you can use that as a unique identifier. So if i am correct then no, you don’t need object url’s.

‌

[xmf]

Is it possible to run an operator or a function right after diffeomorphic daz importer finishes the import? But without editing the importer, so like an event or a way to hook into it.

[Alessandro Padovani]

note for Thomas. Most daz users leave the default path for the daz library, in this case there’s no need to search for the textures since the path is the same. We just need to save the palette with absolute paths that’s an option when we save the blender file. Then some daz advanced users like to change the defaults and add more libraries, in this case yes it is also required to search for the textures. Note that this is true for any blender scene we want to share that is using the daz library.

@xmf If there’s nothing to add this is resolved with commit b7aafc4, for what it can be implemented. Please note that a daz figure is a single object so it makes sense for the palette to address a single object instead of the whole scene. Also this avoids name clashes.

[xmf]

Well, like I already asked, is it possible to hook into daz import process ? If not, can you make it possible so that I have a possibility of implementing my idea on my own without need to modify daz importer?

[Alessandro Padovani]

Issue #1276 was marked as a duplicate of this issue.

[Alessandro Padovani]

Resolved with #1276.