Creating multiple graphs and then building overwrites base manifest

[Matt D]

If I build one graph and then another, the base .manifest file (In my case windows.manifest) is overwritten, thus loosing the information about the other bundles. To reproduce, just build one bundle and then the other. Open the manifest file (Windows/OSX.manifest) and then you should just see one asset bundle under the AssetBundleInfo

[Hiroki Omae]

Thank you for reporting. This is currently by design - and it would happen per BundleBuilder, not per graph (so it may even happen in one graph)

My recommendation from operational standpoint is not mix outputs of BundleBuilders in the same directory, but BundleBuilders should also have name option for base bundle.

[Hiroki Omae]

https://trello.com/c/hIdfEgyk/184-creating-multiple-graphs-and-then-building-overwrites-base-manifest

[Matt D]

Hmmm it's a little weird. Shouldn't it look up the root manifest and append the data if it doesn't exist?

[Hiroki Omae]

@DWSMatt It is not looking up root manifest because each BundleBuilder nodes are build on a unique directory. Also, having 2 BundleBuilders in a graph means that you are building asset bundles in 2 different settings (compressions, type trees, etc). I am not sure if they can be mixed into one directory and share manifest safely. Mixing of bundles in different configurations (and yet share root manifest) need some discussions inside asset bundle team.

[Paul Stapley]

I've just discovered this amazing tool and I thought it was too good to be true.

The root-manifest (iOS + iOS.manifest) should be just that, the file which contains all information for all latest hashed bundles. Without this support then I'm left with many separate folders (one per bundle) each containing it's own root-manifest?? I agree with Matthew above, this is an extremely serious design flaw.

Can someone on the bundle team update this bug with an update pls.

[Hiroki Omae]

@PaulStapley Thank you for the feedback. I am aware that this is one big concern.

In current API of AssetBundle build pipeline, the premise is that you will only build all asset bundles at once to get correct root manifest. In Graph Tool, that is directly associated with BundleBuilder node. Base .manifest files are updated correctly within the set of BundleBuilder node. This is not the limitation that Graph Tool is introducing because it is coming from base API, tool is only allowing you to have option to build multiple sets of bundles, such as build with different compression settings - which obviously, you should not mix them in the same directory.

AssetBundle team is currently working on new AssetBundle Build pipeline APIs, that we have a chance to remove this restriction.

Weaving this limitation in current AssetBundle build API will require a design change that all graphs always must be built at once, at least with ones include BundleBuilder node. That is much more complicated and also slower than having one graph to build assetbundles - I am sure no one wants that.

[Paul Stapley]

Thanks for the quick response Hiroki.

Wouldn't a very easy/quick fix be to scan all the BundleBuilders to match the bundles with the same settings and then under the hood call BuildPipeline.BuildAssetBundles with the same options?

The reason this is so important is because once my game is live I can currently add/append bundles which my players automatically download. Building bundles into separate folders with separate root-manifests completely breaks this support. To use this tool I would have to have my own manifest file which contains a list of all the root-manifests (in separate folders) that this tool generates - meaning your root-manifest is no longer a root-manifest but just another file I have to manage and maintain.

I strongly urge your team to not only build great tools such as this but to also try to use them in a real-life game/product and you'll quickly understand what your clients actually need.

Thanks again for the quick response, I'm not holding my breath that you'll resolve this quickly so I'm off back to using the old BuildPipeline.BuildAssetBundles way to build my bundles.

[Hiroki Omae]

I have raised this issue to critical.

@PaulStapley I would like to be clear that what you can achieve with BuildPipeline.BuildAssetBundles is completely achievable with graph tool and you will not need multiple BundleBuilders to build a variety of different asset bundles. If you need your root-manifests to be intact and keep updated, you can do so by putting everything about assetbundles into one graph, and connect all bundle configurations to one BundleBuilder node. You can create folder structures by introducing "/" in bundle configuration name.

Problem arises only when:

1. Build asset bundles using multiple BundleBuilder nodes
2. Export them into the same directory, and expect them to share dependency information each other.

This expectation is something I have to say "That's not how things work by design, so don't do it" - but when the tool supported multiple graphs, the tool made it looks like you can do it.

So, before just jumping onto issues on the surface, I would like go bit deeper to identify real problem - in my understanding, the question is "Why do you want to use multiple graphs to build bundles?"

Here is two reasons I can think of:

• A. If you try to put all in one graph to build all bundles at once, graph goes too big and less useful.
• B. Sometimes you want to perform build for small sub-group of your bundles for better development iteration.

Here are solutions I can think of for each:

• Solution for A: Provide better UI experience for bundle configuration that one-uber-graph or multiple graphs doesn't matter anymore. (zoom in/out, structured graphs with sub-graphs, etc)
• Solution for B: Make that doable, or rather, remove necessity of building bundles during development time completely.

Asset Bundle team is working hard for solve for B - which will require some core API changes in Unity.
I need to pay some blood/sweat/tears for solution A - but I would like to be sure that I understand the issue correctly here.

It would be very helpful for me if you can share your thoughts @PaulStapley @DWSMatt, especially for the reason why you want to use multiple graphs to build bundles. Thanks!

[Paul Stapley]

Thanks for the detailed response.

I was using multiple graphs because that's just the way I assumed the tool was meant to be used and it really does speed up iteration time like you mention above. I thought that if the build bundles settings matched across bundles it would generate a single root-manifest by merging their final output into one single root-manifest.

For example, my game has 30+ units each of which are contained in their own bundles and all are built using the same BuildBundler settings (deterministic + hashed). Therefore, I hoped/assumed I could simply build them individually rather than all 30+ at one time. The root-manifest binary file (iOS) would then be patched with the modified individual bundle binary.

Thanks, I really appreciate the time you've taken to respond. Regards, Paul.

[Matt D]

We use multiple graphs simply for organisation. We export our items to different "categories" of objects needed, so each of these are their own graphs

• Player
• Entities
• Items
• Environment

This ensures that the bundles themselves don't get too big and are still organised. It really speeds up iteration, like @PaulStapley mentioned - if I make a change to environmental asset A's texture, I can just rebuild the bundle and it will work in the player I built yesterday. Also, I wanted a graph for each bundle

I was more concerned about the root manifest file for editor purposes - instead of loading the bundles during playtime, it is much easier to simply lookup the asset bundle manifest and find the path to the asset and then called AssetDatabase.LoadAssetAtPath to load it. Unfortunately (this seems inherent) you can only access asset bundle manifests after loading the asset bundle, which seems weird (correct me if I am doing it wrong).

Thanks for taking the time to reply @homae - I speak for everyone involved with AGBT that we really appreciate your support and time that you take to help us :)

[Hiroki Omae]

I will mark this issue as "on hold" - I have worked on a few ideas pursuing Solution B direction but couldn't come up with good solution.

I will go for Solution A for this issue when necessary APIs such as zooming comes to Unity GUI API.