Warn the user if creating a project (or even editing a subscription) against an old (and potentially/likely stale) org metadata cache

[René Görgens]

Hi Scott,

Surprisingly, I’m seeing a mixture of stale metadata and fresh metadata in the module configuration – after updating the cache with “Update Now”.

• This is a report folder. The selected metadata are stale whereas the unselected metadata are fresh (they are “duplicated”). The stale metadata don’t go away when doing additional cache refreshes, or reopening the project, or restarting the IDE, or clearing IDEA file system cache and local history.

‌

• Additionally, I’m seeing the following separate subnode within Report (stale meta.xml for the report folder)

‌

• Interestingly, I’m only seeing fresh metadata in the retrieve dialogue, although the same cache timestamp is being used:

‌

• Note: I did start this MDAPI project by selecting the report folder (not individual reports inside the folder), however without having refreshed the cache (which was dated 04/11 or so). Nothing was initially retrieved but package.xml has the stale metadata. I guess this is why the stale metadata are showing up in the module config and don’t go away when refreshing the connection.

  • => Yes, the stale metadata are gone from the module config after cleaning package.xml manually.

So in conclusion I’m not sure there’s a defect here, but it would for sure be helpful to be reminded of the cache age during project creation in order to avoid this scenario.

[Scott Wells]

The org metadata cache is something maintained by IC2 and isn’t going to be cleared/rebuilt by the IDE’s native file-based indices. You can enable debug logging for it by adding the following to Help > Diagnostic Tools > Debug Log Settings:

#com.illuminatedcloud.intellij.configCache.FilePropertiesCache

and then using the Refresh toolbar button or Update now link. I can scan the resulting idea.log, but it might be useful for you to make a first pass to correlate what you see incorrectly in these dialogs vs. what gets shown in that log based on the org’s reported metadata. If you could clearly designate things that are present in the dialogs but not in the debug log, that would help me from a diagnostic perspective so I can figure out what might be happening.

[Scott Wells]

Just saw your update. Okay, so it seems that it’s because the project was created based on a stale org metadata list. Makes sense. I’ll update this as an enhancement request to warn the user if creating a project against an org list older than some reasonable span of time.

[René Görgens]

Thanks for your help. It turned out to be stale metadata in package.xml which were (quite logically?) shown in the module config dialogue. After cleaning package.xml, these were all gone from the module config.

So this is essentially a user error, forgetting to refresh the connection prior to configuring the project initially, resulting in a void retrieve but updating package.xml with the stale entries nonetheless.

So in conclusion I’m not sure there’s a defect here, but it would for sure be helpful to be reminded of the cache age during project creation in order to avoid this scenario.

[René Görgens]

Thanks for seeing the potential of this enhancement.

Upon reflection, isn’t any remote-based refactoring likely to cause this kind of issue, at least with MDAPI? By which mean to say,

• If you’re refactoring locally, IC will keep track of it and replicate changes in the remote (create/delete org metadata accordingly)
• If you’re using SFDX with source tracking (which I haven’t done yet), then barring .forceignore you’ll have correct replication of changes from remote to local. Or do you? Is the remote version control that good?
• So doing remote changes with MDAPI is inviting this sort of problem, and unless you manipulate package.xml to reflect your remote refactoring, it’s not going to go away

Where it gets complicated is if the metadata type does not support API renaming e.g. Flow. You save a new flow and discard the old one. I don’t reckon source tracking can cope with that.

[Scott Wells]

IC2 tracks references to top-level Apex classes and triggers in package.xml files, so when you rename one locally in IC2, the rename is reflected properly in the package.xml file in addition to IC2 prompting you to update the org with the appropriate changes. Or am I misunderstanding your concern? I’ll say this, I’ve been using that feature regularly since I first implemented it and haven’t seen any issues. Of course, only the project’s connected org is updated, so if you’re working on something that gets checked into version control and deployed into other organizations that aren’t ephemeral in nature (i.e., scratch orgs created for CI/CD), you’ll need to take those changes into account. On my previous projects we also maintained destructiveChanges*.xml files so that deletes on a dev/feature branch were propagated into other environments.

[René Görgens]

Exactly Scott, I have experienced these features: when I refactor locally (even without version control), I’m at ease because IC tracks it and prompts me to update the org (my first bullet above).

However when another developer has intervened on the same org without version control being in place, then that’s a recipe for problems, at least with MDAPI (manual diffs at best). The problem could be mitigated with SFDX + source tracking activated, but I have to try and fully understand that yet.

The eventual solution for a collaboration scenario is of course version control, which is what we’re implementing on our project.

‌

[Scott Wells]

Yeah, that’s not a problem that IC2 seeks to solve, at least not right now. There are so many potential issues that can result from collaborative development against the same organization without using version control capturing and managing the proverbial single source of truth. IC2’s conflict detection features can help guard against last-writer-wins types of issues, but it’s not a silver bullet.

This problem generally arises in sandboxes since folks don’t (or at least shouldn’t) be actively working against production, and other org types--DE orgs and scratch orgs--are generally per-developer. In those situations a source-tracked sandbox would be the right way to go, particularly if VCS isn’t being used.

But yeah, best practices would dictate that the specific subset of metadata that constitutes the customizations and extensions required to take any org of the appropriate shape and update it to represent the team’s goals should be maintained in a version control system.