Nice Tool but Slow

[Graeme Scott]

We have an existing project that is quite large and was not using asset bundles (but we have a need to get them working)

I looked into various options for assigning bundles, this seems to be the best.

But we are running into an issue where the graph takes > 10 mins to process, I did a profile in the editor with "Deep Profile" on and discovered that AssetDatabase.GetDependencies seems to be extremely slow with our data set.

I think this data needs to be cached either by Unity or the GraphTool

I am going to see if I can replace these calls with some kind of caching system, that uses AssetPostprocessor to keep track of changes etc.,

Not sure it will work but the current implementation is unworkable for us..

[Hiroki Omae]

Thank you for reporting @graemezengami .

To understand the issue better, can you share more info about your project?

• Size of Project (how many gigabytes)
• How many assets are in your project
• How many assets are your graph is processing for building asset bundles
• How many assetbundles are you building

Also following info will be very helpful.

• screenshots or json data of your actual graph will be very helpful.
• screenshots of the profiler

Thanks!

[Hiroki Omae]

Currently AssetDatabase.GetDependencies is used in:

• Extract Shared Assets
• Assert Unwanted Assets In Bundle

Which node are you using in your graph?

[Graeme Scott]

I will have to get back to you on some of these questions, we are busy with submission process for one of our games..

I am using "Extract Shared Assets", but I notice delays on other nodes like "Load From Directory" etc.

When I get back into this I will give more information about our usage patterns

Cheers Graeme

[Graeme Scott]

Had a little more time to look into this, our project uses ~2.5GB data and has ~6500 files.

The "Load From", it was because of our heavy usage of ScriptableObjects. The issue is caused by AssetDatabase.GetMainAssetTypeAtPath not returning the correct type for ScriptableObject derived classes, so the utility function TypeUtility.GetMainAssetTypeAtPath then loads the asset in order to get the correct type. Which takes a large amount of time in our project. I worked around this since in the editor you can search for these in the project view "t:SomeClass". I used the AssetDatabase.FindAssets function to get me the data without loading in the files.

I am still experimenting with the "Extract Shared Assets" and a AssetDatabase.GetDependencies cache, to see if I can speed this up. As again with some of our ScriptableObjects (used to hold level data) can have quite a lot of references, and takes a long time to calculate. I am basically trying to cache each file dependancies and using AssetDatabase.GetAssetDependencyHash to determine if I need to re-cache them. Seems to be partially working in my testbed, I will then replace the calls in the AssetGraph and see how it goes..

Thanks Graeme

[Hiroki Omae]

@graemezengami Thanks for the update! I am quite pessimistic at this point about caching dependencies strategies can benefit performance because it is difficult to predict change - i.e. When I have a model who reference a material, then there is a change in texture reference of the material, graph can not effectively detect model's dependency change because it does not trigger model's modification.

Instead, I am thinking of adding auto update toggle for "Extract Shared Assets" where you can turn off automatic recalculation in every Prepare. This could at least improve graph authoring experience. I am interested to know if that solves your problem.

It's good to hear that you have found workaround for Loading node performance issue. For AssetDatabase.GetMainAssetTypeAtPath() not returning correct types for scriptable object - I believe that behavior is improved in 2017.1 so updating version could benefit the performance as well.

[Graeme Scott]

it seems that AssetDatabase.GetMainAssetTypeAtPath and the work around in TypeUtility.GetMainAssetTypeAtPath is causing problems all over the place...

[Graeme Scott]

Unfortunately I am not sure we will be able to upgrade to 2017

[Graeme Scott]

BTW from my test of the asset cache, it does indeed account for changes to dependencies. I am using AssetDatabase.GetAssetDependencyHash to determine if the dependencies for an asset have changed.

I don't cache all dependencies for an asset only immediate dependencies, when recursing I then look up the dependency and do the same checks..

The only thing I noticed is that this HASH is only updated after a save, and even AssetDatabase.GetDependencies will not pick up the new dependency until a save is performed..

I am not sure if this will reduce the times by enough, but on my assets AssetDatabase.GetAssetDependency can take quite some time...

[Graeme Scott]

The other thing I noticed while looking into performance was that ExtractSharedAssets.CollectDependencies is using a List<> on the collector. Then using Contains and Sorting, with my usage patterns this is really slow. I am looking at changing the List<> to HashSet<> to try and minimise the time here.

[Graeme Scott]

I have also noticed with our usage patterns, that the AssetBundleBrowser is quite slow generating the bundles to display. I have not looked into why this might be yet!

[Hiroki Omae]

This project is now migrated to Github. Open issues will be addressed in following page:

https://github.com/Unity-Technologies/AssetGraph/issues