Possibility of Caching Metadata Objects?

[Henry Zhao]

Hi Scott, back with another minor gripe.

Every time I want to deploy or retrieve components, IC is forcing this Populating Contents dialogue on me. My org isn’t necessarily huge, maybe ~1000 different metadata, but this Populating Contents window is taking excessively long for every single operation.

As I’m writing this, I believe my IDE has flat out hung, as the window has been running for 20+ minutes.

My ask is: Is it possible to cache the information so we are not listing metadata objects every single operation? I understand this will result in stale metadata, but I would much rather have the option of refreshing manually in the retrieve/deploy UI, than waiting 20sec-2min every operation.

‌

[Scott Wells]

Henry, IC actually does cache the metadata list from the organization pretty aggressively, and for a while it also uses up to 25 threads to list the metadata from the org in parallel. As a result, the behavior you're describing sounds unexpected. Can you please add the following to Help>Debug Log Settings, reproduce the behavior, and provide the resulting idea.log* file(s) from the time span of the reproduction?

#com.illuminatedcloud.intellij.builder.ForceComBuilder
#com.illuminatedcloud.intellij.builder.ForceComBuilderUtil

Hopefully that will show me what's going on and where the time is being spent. It will also hopefully show me why it's having to do this every time when it generally should have this information in a cache until you explicitly tell IC to list the org's metadata again.

[Henry Zhao]

Hi Scott. I’ll leave these additional traces in for now. The issue only happens once in a while, usually in the morning when I’m doing a “cold boot” and opening up my projects. I’ll report back with the logs the next time this happens.

I am confused however, if IC is already caching the metadata list, how come every time I open the deployment/retrieve UI, it’s still showing me that window of Listing Metadata Objects? Is it just reading from the cache in that case?

[Scott Wells]

That's what I'd like to see, Henry. It's been a while since I've looked closely, but perhaps it has a persistent cache for the subscription editor but only an in-memory cache for other operations to ensure they're kept as up-to-date if possible. If that's the case, a persistent cache could certainly be put in place for everything with an option to refresh it explicitly. Before I go that route, though, I'd like to see a log to understand why it's taking so very long to enumerate all metadata in your org even with queries executing in parallel.

[Piotr Kożuchowski]

@Scott Wells Btw. I wonder if it would be possible to skip “Listing metadata” for “Context” retrieve?
When I want to check single file from other sandbox (ex. UAT) it looks like this:

1. Right Click and retrieve
2. Wait for loading of full metadata list
3. Change connection to UAT
4. Wait for loading of full metadata list
5. Retrieve

Since I don’t need any list other than context files, I’d be fine if Retrieve was executed right away, and if the file is not there - it can notify me afterwards that 0 files were retrieved.

‌

[Scott Wells]

This was delivered with 2.1.2.0. Here's the relevant portion of the release notes:

Centralized, persistent org metadata definition cache - Org metadata definitions are now cached persistently across sessions, and this cache is used consistently throughout Illuminated Cloud to avoid unnecessary (and sometimes expensive) metadata enumeration. This cache can be explicitly refreshed using the corresponding Refresh toolbar buttons in the deploy/retrieve/delete dialog and the metadata subscription editor. The cache population date is displayed clearly in both of these locations. NOTE: All other persistent org metadata definition caches have been removed and replaced. This includes the skeletal metadata definition information stored as part of a Selected metadata subscription in the .iml file. The next time that a Selected metadata subscription is updated, only the selected items will be included in the .iml file. This makes for a much more compact .iml file that represents exactly the metadata included in the subscription. Similarly, if a subscription type other than Selected is used, the entire Manifest section will be removed from the .iml file.