Save metadata selection back to package.xml

[Nathen Drees]

When configuring a project, we would like the metadata selected to be included in the project to be saved back to the package.xml. For example, if we select All/Package, we would like the package.xml file to be updated with *'s where possible, and list out the other metadata. If we choose the 'Selected' route, it should update with out selections.

Most other tools in the SF world are driven off the package.xml, so writing back to this file will increase interoperability between the IDE and other tools (like build severs).

I'm not sure the 'Package.xml' option makes sense anymore if everything is being driven off the package.xml file? It might only be useful when creating a new project?

[Scott Wells]

Nathen, then you retrieve metadata through any of the configured options, the left-hand side of the directory diff should include a package.xml based on whatever mechanism you've selected. You can then just copy/sync that file into your local file source root. Aside from the timing, is that what you're wanting?

[Nathen Drees]

Not exactly. The problem with using the package.xml that's returned from the server is that if you have no metadata components for a metadata type, it just doesn't return anything for that component in the xml, instead of leaving it in there.

Example: on my current project, we have no AccountCriteriaBasedSharingRules defined currently, but I want that selection to stay in the package.xml in case one of our Analysts adds one later. However, when I refresh from the server, the changes I see being made to the package xml are:

If i'm reading this right, it's saying it's going to remove the AccountCriteriaBasedShairingRule if I were to take what was returned from the server. To work aroudn this, I would have to do a manual merge of the file every time we retrieve it, which is not ideal.

In our setup, we have an integration server that periodically uses the package.xml file to poll salesforce and check all of our metadata in to source control, for disaster recovery purposes (beyond just technical ones, like an analyst going rogue, client suddenly revoking access to the org, etc).

This is why we would like our selection to be saved back to the package.xml file, regardless of what's actually in SF. Then if an analyst later adds some configuration, the integration server will later pick up the changes and check them in.

[Scott Wells]

Gotcha. So you're basically saying that the "Selected" option would basically become an editor for the module's package.xml file itself, correct?

[Nathen Drees]

Essentially, with the All/Package option being the same thing as clicking all of the check boxes.

[Scott Wells]

Right. Good qualification. Makes sense. Thanks for clarifying!

[Nathen Drees]

Scott -

Do you have a road map in mind for this? This is one of our larger pain points now, as it's messing with our other tools.

[Scott Wells]

Nathen, let me take a look at this again over the next few days and I'll let you know my thoughts on when/how this will manifest.

[Scott Wells]

Okay, I looked at my current backlog and chewed on this a bit. I have one more major feature I want to crank out ASAP and think I can slot this in directly behind that. Of course bug fixes that are affecting users will also take priority. Based on that plan, I think I should be able to turn to this in the next couple of weeks.

In the interim, is using the "Package.xml" option with your own package.xml file properly populated a reasonable workaround?

[Scott Wells]

Nathen, this is still on my near-term agenda. I'm going to be showing IC at Dreamforce and won't be making many changes to it other than small bug fixes and refinements leading up to that. Once I get back I plan to concentrate on a set of deployment/retrieval enhancements including this one. Just didn't want you thinking it had dropped off my radar!

[Scott Wells]

Nathen, just thought I'd let you know that I'm getting started on this now. Should have something out soon!

[Scott Wells]

Hmmm...okay, so I started implementation of a first cut at this and, as I was testing it to validate the behavior, I found that I'm not seeing the original stated issue. Let me tell you what I've done and you can either tell me what I've done wrong, or perhaps Salesforce has changed the behavior to be what you want.

1. I updated my explicit metadata selection to include a few metadata types that have no actual rows in my Salesforce SF org, e.g., ApexComponent and ApexTrigger.
2. I used IC to retrieve metadata from the server.
3. I looked at the retrieved package.xml and it does in fact contain the appropriate wildcard entries for those even though no components or triggers were returned.

Then I decided to see if perhaps it might be something special about the particular metadata types you included in your own example, AccountCriteriaBasedSharingRule and AccountOwnerSharingRule. In my second test, I did the following:

1. I switched metadata selection to use my own package.xml and added wildcard entries for both of those types.
2. I used IC to retrieve metadata from the server.
3. I looked at the retrieved package.xml and it does in fact contain the appropriate wildcard entries for those two types.

I ran a third test where I selected the All option and did a retrieve, and the returned package.xml had everything including wildcard entries for metadata types that have no entries in the org. As far as I can tell, it's basically returning the package.xml that I'm sending to it based on the local selection.

Do you mind checking this out on your end? If you can reproduce it and the steps I've followed above don't match yours, can you provide corrected steps? Thanks!

[Nathen Drees]

Scott -

I'm still able to reproduce this issue. I've included the original package.xml file i have in the project, as well as the one that is returned from Salesforce after retrieving metadata. You can see that some of the entries are removed, even though they are valid metadata components.

I have my metadata selection set to all/package, so it should just pull down everything.

Is this just an issue with the package.xml file I generated before? I used https://packagebuilder.herokuapp.com/ to generate it.

[Scott Wells]

Thanks for the concrete example, Nathen. As odd as it may sound, I'm actually kind of glad that it's still reproducible because I actually have the fix for it locally.

For what it's worth, I'm not going the full route of consolidating the three options into an effective package.xml editor. Some other users have chimed in saying that they like the explicit option to use their own package.xml. Instead what I'm doing is (based on a configurable setting) replacing the server-provided package.xml with the one I used when retrieving metadata which is itself based on the current selection. Should achieve the same overall goal...

I'll see if I can get this wrapped up tonight or over the weekend. Hopefully what you've provided will allow me to test it adequately.

[Scott Wells]

Looking at this now. I placed your original-package.xml into one of my projects as package.xml, set the module up for Package.xml selection, and retrieved metadata. There were no differences at all between the one that came down in the retrieval package and the one that I have locally. Obviously that doesn't match what you're seeing, though, so I'm not saying it's not an issue...just that I'm having trouble reproducing it.

It could even be a difference in orgs. We've seen quite different behavior between orgs created a few years back and upgraded over time vs. newly-created orgs. For example, newer orgs tend to return metadata XML in a stable order, but some older orgs do not. Some orgs include "<deprecated>true|false</deprecated>" in custom field definition in *.object files and some do not. Honestly it drives me crazy because when we create new versions of our managed packages, we like to retrieve metadata from the packaging org and compare it to the metadata in the source repository to verify that the differences are exactly what we'd expect, but in some orgs we get additional differences like these and have to wade through them.

So having said that, I definitely understand the consternation that you're having with getting back a package.xml that doesn't even come close to matching the local one! Since I already have a working solution for this, let me see if I can get a test build to you so you can verify whether it helps address the issue or not. If not, it'll provide fast feedback so I can course-correct early; if so, I'll go ahead and polish it up for the next general release.

I'll see if I can post a test build in the next hour or so.

[Scott Wells]

Nathen, I'm going to send you a direct message on here with links to the test build and details about the changes. Please provide feedback on whether it addresses the issue adequately for you.

[Scott Wells]

Resolving as won't fix after discussions with Nathen.