New items are not added to metadata subscription automatically

[René Görgens]

Context: non-SFDX, non source control project

After experiencing issues with the metadata subscription setting “Prompt”, I did the following:

• Switched setting to “Always Update”
• Emptied .iml file content (all <MetadataEntry></MetadataEntry> nodes)
• Selected “Add to Metadata Subscription” on src

Result: Correct .iml file, all subscriptions showing in Retrieve Metadata.

However when I then add additional metadata (example: workflow field updates):

• These are not automatically added to the metadata subscription (.iml file) despite the “Always Update” setting. They appear in light blue and aren’t selected when opening Retrieve Metadata, whereas the subscribed items are in bold blue and selected
• They are effectively added to the .iml file when I manually select “Add to Metadata Subscription“ on the respective node

Is that a defect or not? Thanks in advance

[Scott Wells]

Hi. What types of issues were you experiencing with Prompt as an option? I’m wondering if there’s something going over in general when that’s set to either Prompt or Always Update for you.

Can you please add the following to Help>Diagnostic Tools>Debug Log Settings and reproduce the issue?

#com.illuminatedcloud.intellij.settings.module.ModuleContentsSynchronizationListener

Then please either attach or email your idea.log file for review.

[René Görgens]

Hi Scott,

Thanks for the quick follow-up. I found the behaviour with Prompt to be somewhat unreliable but that’s from (recent) memory.

I tried to reproduce the issue even before enabling the debug log setting and was unable to do so, but instead experienced the reverse:

• Subscribed to an additional WorkflowFieldUpdate and actually got a message that the metadata subscription was updated (I restarted the IDE since I had the issue)
• Checked Retrieve Metadata: the new item is in bold blue, thus looking well subscribed
• However, it’s definitely not in the .iml file, even after multiple reloads from disk. It just took an eternity to materialise in there

Enabling the debug log setting now… OK

• Bingo, I was able to reproduce the issue
• I sent you the log file + screenshot by email to [e-mail]

Thanks

‌

[Scott Wells]

Okay, so just to clarify, you are seeing it added to the .iml file, but it doesn’t happen instantaneously, correct? Because that’s to be expected. That’s an IDE project configuration file, and it flushes it to disk on its own schedule…kind of. Under the default configuration, if the IDE process loses focus entirely, it should flush its config files to disk. Also, IC2 tells the IDE to save all files to disk before it performs certain operations. Note that this is different from if you use a package.xml-based subscription where IC2 is 100% responsible for writing to the file and you’ll see subscription updates happen immediately.

Do you see the .iml file updated consistently if you switch away from the IDE process completely?

For what it’s worth, I see the addition happening in the provided log:

2022-10-12 21:58:47,261 [ 109091]   INFO - #c.i.i.s.m.ModuleContentsSynchronizationListener - The metadata subscription for module 'CorCRMDEV' was updated:

WorkflowFieldUpdate
• Case.BIL_UpdatewhenApproved [ADD

‌

[René Görgens]

I’m seeing significant latencies (30-60 seconds or more?) on the following:

• Emptied .iml file content (all <MetadataEntry></MetadataEntry> nodes) because it was having lots of removed metadata
• Selected “Add to Metadata Subscription” on src
• Being able to see the updated .iml file

Reload from disk didn’t show an updated file. I resorted to closing and reopening the project, that worked.

This is on a new laptop with Ryzen Pro 5 and 16 GB of RAM

[Scott Wells]

Well, a few things of note. First, you really don’t want to make manual modifications to the IDE’s config files for the same reason I stated in my previous response. The IDE has its own cache for those and, while it should be able to reconcile such explicit changes to the raw file with what it has in-memory, you’re playing with fire. Second, the IDE is going to flush those caches to disk on its own schedule. That doesn’t mean that the in-memory version doesn’t represent the current/correct state, though. If IC2 shows a notification that the subscription has been updated and you open its subscription editor and don’t see that reflected there, that’s a bug. Similarly if IC2 misses a file addition/removal that should have been processed, that too is a bug. But if the .iml file just isn’t being updated for some period of time, that’s just how it’s going to work given that a Selected metadata subscription uses the IDE’s configuration files for its on-disk persistent storage. The alternative would be to change to a Package.xml subscription where IC2 is 100% responsible for writing changes to the physical file.

Apologies if I’m missing something, but it sounds like things are working properly based on the description above. Is that not correct?

[René Görgens]

Do you see the .iml file updated consistently if you switch away from the IDE process completely?

=> Yes indeed

[René Görgens]

Apologies if I’m missing something, but it sounds like things are working properly based on the description above. Is that not correct?

=> Thanks a lot for your observations. On the whole, the explanation covers the behaviours noted in the context of this issue, thanks for educating me

Takeaways:

• Old/renamed metadata sometimes remain in the .iml file, not sure if that’s actually an issue, but I now know how to fix it using the workaround(s)

Remaining question:

• Adding a metadata subscription, although resulting in a pop-up, didn’t show correctly when bringing up Retrieve Metadata subsequently, even after several minutes. I checked this again:

  • Added a new metadata subscription and had a message
  • Switched away focus from the IDE
  • Brought up Retrieve Metadata: new item listed in light blue and not checked
  • Restarted the IDE and it is still not checked
  • I’m having a third variation now, the new metadata subscription is definitely in the .iml file but definitely not showing up visually under Retrieve Metadata

For me it’s not really working, but if it’s IDE quirks then you can’t do anything and this should be closed.

[Scott Wells]

Two items stand out as potential bugs in your summary:

1. Old/renamed metadata sometimes remains in the .iml file - Can you provide specific steps to put things in this state? That shouldn’t be the case, and if it is happening, it’s a bug.
2. Added…and had a message => New item not bold or checked - Let me try to reproduce this tomorrow a bit because that also sounds like a bug if so.

‌

[René Görgens]

Thanks a lot for your help today. I’ll have another go tomorrow to figure this out better

[René Görgens]

Good morning Scott,

Typology of issues:

1. Despite metadata subscription setting “Always Update”, subsequent metadata retrieves are not done automatically

   • Metadata subscription messages are not always appearing after retrieve (repectively retrieve for merge)
   • When checking the subscription using Retrieve Metadata, the subscription is not visible (light blue and unchecked instead of bold blue and checked). NB: It works for the initial item(s) in a new project but fails for subsequent additions.
   • When “Add to Metadata Subscription” is invoked, the missing subscription is added and visible in Retrieve Metadata

I was able to reproduce the above issue:

• Create a new empty project and add 1 metadata item (ex: WorkflowFieldUpdate)

  • Subscription done and visible immediately

• Do Retrieve Metadata and select an additional item (ex: WorkflowFieldUpdate)

  • Subscription not done, no message after retrieval

I’ll look into the other types later.

I’ll send the log file to [e-mail]

[René Görgens]

Typology of issues:

2. When deleting metadata from a project, the entries in the .iml file remain, even upon project reopening

I was able to reproduce the above issue:

• Retrieve a metadata element (e.g. a Report)
• Delete the same metadata element locally (w/o deleting it remotely)
• Close the project
• Reopen the project
• The element (Report) is still in the .iml file

The above issue (2.) has been tested outside of the corporate network in which I’m working:

• Metadata subscribes are much faster and more reliable outside the corporate network. I could not reproduce the above issue (1.) outside the corporate network. So far, with the option “Always Update” enabled, outside the corporate network I’ve seen an instant metadata subscription message and Retrieve Metadata shows the subscription as selected and bold blue.

Summary:

• Issue (1.) seems to only occur inside the corporate network
• Issue (2.) occurs both inside the corporate network and outside

‌

[René Görgens]

Regarding the point: Metadata subscribes are much faster and more reliable outside the corporate network.

• I found a potential explanation: My JetBrains workspace wasn’t located in the folder which is exempt from virus scanning. I’ll try to place it there and see what happens.

‌

[Scott Wells]

Ah, that’s interesting. Note that for optimal performance, there are a few things that should be excluded from antivirus in addition to the project directories:

https://intellij-support.jetbrains.com/hc/en-us/community/posts/360003273799-IDEA-Recommended-virus-checker-configuration-and-exclusions

[René Görgens]

When locally deleting metadata from a project, the entries in the .iml file remain

[Scott Wells]

Okay, I was finally able to reproduce this and have found the root cause. I’m testing a fix now. It’s a bit late to try to get it into tomorrow’s build, but it should go into next week’s build no problem.

[Scott Wells]

Delivered in 2.2.4.4.