Easy way to exclude managed Metadata from retreival

[Christopher Tufts]

This isn’t an issue, but more of a question/feature request.

I am leveraging some tools for CICD that I am running into issues excluding the managed package stuff from. In reality I don’t any of them in my repo

Is there a way to easily say get me everything BUT managed package items from a retrieval scope?

this would also nee to consider the fact that I can still get all the type-ahead IDE functionality without actually subscribing/retrieving to those things correct?

‌

thanks

[Scott Wells]

Hi, Chris. Let me start by repeating back my understanding of the question(s) and even what comes to mind as a result just to be sure we're having the same conversation.

IC2 supports retrieval of metadata from installed managed packages as long as they are considered subscriber-editable. At present there is no way to filter the build options dialog used for bulk metadata deployment/retrieval/removal to include/exclude metadata from installed managed packages. If I understand the main question properly, you're asking for some way to do that, correct?

As for the second question about code completion for types, objects, fields, etc., from installed managed packages, that's actually all based on the contents of the generated offline symbol table and is unrelated to whether you have or have not retrieved metadata from those packages.

If I've understood this properly, then the core request is to have some way of filtering the build options dialog based on package status. If what I said above was a total swing-and-a-miss, please feel free to correct me and we'll proceed from there.

[Christopher Tufts]

So,

Second question you verified my understanding of how everything works which is perfect.

The first question isn’t about pulling managed package metadata, its about NOT pulling managed package metadata, sorry reading it back now thats terribly unclear above…

I would like an easy way, for instance, to say get ALL my objects, but exclude managed packages without having to go cherry pick all the new components from my development work.

thanks

[Scott Wells]

Yeah, I think we're saying the same thing. I was just suggesting that perhaps it's just a matter of adding another filter checkbox to the build options dialog for Packaged alongside Local Only, Subscribed Only, etc., that, when checked (default) would include subscriber-editable metadata from installed packaged, and when unchecked would not include those items. That meets the need, right?

[Christopher Tufts]

ah I understand, yes that would help

thanks

[Scott Wells]

Chris, I've been thinking through this a bit more and am trying to make sure that I understand the full use case. The contents of the build options dialog are generally filtered by the configured metadata subscription, i.e., the subset of org metadata that is considered part of the working project. The metadata subscription is configured either via Illuminated Cloud>Configure Module or, in the case of a Package.xml subscription, explicitly by editing the corresponding package.xml file.

So it occurred to me that perhaps what's needed is the ability to decide whether to include/exclude subscriber-editable items from installed managed packages in the subscription editor since that will ultimately determine what's shown in the build options dialog (at least when the Subscribed Only checkbox is selected).

First question for you...what kind of metadata subscription are you using on these projects? It's not All/Package, is it? If so, I recommend you switch from that to one of the other subscription types as that one can cause all kinds of issues. I'm actually strongly considering removing it as an option altogether.

Continuing the thought above, though, things get even more confusing when there are several metadata types that support wildcards and are also subscriber-editable, e.g., CustomApplication. So if you wildcard CustomApplication but don't want to include metadata from installed managed packages, any kind of filtering can get strange.

I'll need to think about this a bit more as it's not quite as straightforward as I originally hoped it would be...

[Christopher Tufts]

It never is straightforward with salesforce!

but yes, I would want a straight up ‘exclude managed package’ and less about subscriber editable (though I am assuming that is the only thing that you have to filter on)

My Problem is this:

My migration tool of choice Gearset can exclude managed packages from a compare when they’re pulling from an org, but cannot when they’re pulling from a repo. thus forcing me to manage it 1 of 2 ways.

1. As you’re suggesting through package.xml
2. Just don’t ever include things that will break deployment in the repo (like things that aren’t editable)

Managing a package xml via git is painful at best as git doesn’t understand the xml structure, so I was trying to take an easier approach, but it would seem we will need to deal with fixing endless merge conflicts in a package xml. which is what it is at the end of the day, I was just hoping I could avoid it.

I do like the approach and think its worth adding something like this into the platform though.

‌

[Scott Wells]

I honestly think that perhaps the best way to do this is to manage the metadata filter via a package.xml file that explicitly includes only the items you care about (ugh...dangling preposition).

I'm confused by your comment about managing a package.xml file in Git, though. I've done that in pretty much every project over the years and, as long as you maintain a stable element ordering (typically alphabetical), you get very useful diffs. And IC2 does help to maintain package.xml files in stable ordering by enabling Illuminated Cloud > Configure Application > Validation and Deployment > Metadata subscription > Sort package.xml entries. Can you elaborate on the issues you've had using a package.xml file with a Git repo? Perhaps I'm misunderstanding.

[Christopher Tufts]

I think this may be much different than any past experiences that I have worked with as for right now, we are all using WS/IC in the past much of the problems came when there was no strategy to keep the package.xml clean. some people would add new things at the end, some at the beginning, and even no one agreeing on tab/spaces, which made it look like IC was editing the entirety of the file.

So I am willing to give it a try here with a much more controlled environment at least.

‌

thanks!

[Scott Wells]

Ah, I see. If you enable the option to keep the package.xml file in a stable order, every time that IC2 updates it, it will reorder things that aren't already in alphabetic order. It will retain any comments as adjacent to the contextual element(s), though. If that would cause an issue, you can disable that option and it will always add to the end. However, that ordering won't match what you get back from Salesforce when retrieving metadata whereas the stable ordering option should always match. In my experiencing maintaining in stable ordering leads to very easy-to-diff versions over time, though it may not match everyone's workflow.

And I'm not going to close/resolve this enhancement. I still think there's something to address here. But it won't be a quick change as I really want to figure out exactly what the change should be...

[Justin Julicher]

Chris, at my previous work we had managed packages and used gearset also. I was able to create a set of regular expression filters that allowed me to exclude managed package metadata from the different lists, however it was not a trivial regular expression using look aheads etc and took me half a day or so to get right. I no longer have access to that but thought I would throw it out there that it is possible to add those filters in Gearset also.

‌

[Christopher Tufts]

Thanks Justin,

I will keep that in my back pocket. I think I would prefer to attempt to keep the repo clean. I finally had success getting a first deployment to an integration org, so I think we just need to keep everything clean moving forward. This hurdle is always the hardest tbh.

‌

thanks for the info though!

[Scott Wells]

Issue tracker grooming. If this is still an issue, please feel free to reopen, ideally with a concrete reproduction scenario.