Retrieval of *active* flows (made in Process Builder)

[Andrew Clunis]

Flows (that is flows, not flowDefinitions) that are marked as inactive can be retrieved in Illuminated Cloud's Retrieve Metadata functionality, but if they are active they do not appear in the list of available metadata on the SF instance.

We tried using the Force.com migration tool (which apparently uses the older metadata API, if that's relevant) to pull them down, and that works.

Considering that active flows are read only, it seems to me that they should still be retrievable, even though they're not publishable.

There are no relevant log entries; no errors or warnings occur.

Thank you!

[Scott Wells]

Thanks, Andrew. Let me try to reproduce this. The only filter I'm putting on metadata presented for retrieval right now is for top-level metadata (vs. child metadata like CustomField) from installed managed packages...at least the only one I'm intentionally using! I'll let you know what I find.

[Andrew Clunis]

Thanks for the prompt reply, Scott. Looking forward to it!

[Scott Wells]

Andrew, can you provide an example of the package.xml that you're using when retrieving with the Force.com Migration Tool? I'm pretty sure I'm reproducing what you're seeing. I've modeled a sample process using Process Builder and activated it, then I've subscribed to that process for both the Flow and FlowDefinition . I've verified that both of these are being specified in the metadata API retrieve request, but I only get back the .flowDefinition file (mostly empty) and not the .flow file. I'm curious about how you're specifying this so that everything you want is coming back. Thanks!

[Scott Wells]

Hmmmm...this looks like it could be related:

https://success.salesforce.com/issues_view?id=a1p300000008ZBfAAM

I'm wondering if this is an issue with the Salesforce metadata API right now. I'll keep investigating.

[Andrew Clunis]

Illuminated Cloud's retrieval package.xml setting is set to Package/All.

Isn't Illuminated Cloud using the tooling API?

Or are you using the metadata API, specifically the "bulk retrieve" endpoint (which seems to be what's broken, according to that salesforce community ticket you linked)?

Apparently, the force.com migration tool is indeed managing to retrieve the flows (using the retrieveUnpackaged ant target, which does not use bulk retrieve API, I believe, but still uses the older Metadata API). We're using a package.xml in which the flows are wildcarded, which works.

[Scott Wells]

Right now IC only uses the Tooling API for Apex classes/triggers and Visualforce pages/components, and only when configured to do so (Settings>Illuminated Cloud>Validation and Deployment). The Tooling API only supports a very limited set of metadata types right now (though it is more than just Apex/VF), so any deployment that includes more than those types automatically uses the Metadata API.

In this case I'm doing a SOAP Metadata API retrieve request specifying unpackaged=true along with the subscribed metadata types/components or, if you've associated your subscription with a development package, singlePackage=true and packageNames={selectedDevelopmentPackageName}. I'm not 100% sure what "bulk retrieve" means, but my guess is that it's using the same API.

[Scott Wells]

Aha! I've figured out what's going on here now. For some reason the name of the Flow metadata is being reported by the metadata API as, for example, Issue280, but when successfully retrieved, the actual metadata name is Issue280-1 and the resulting file is flows/Issue280-1.flow. As a result, a retrieve with the FMT tool using the package.xml:

<?xml version="1.0" encoding="UTF-8"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
    <types>
        <members>*</members>
        <name>Flow</name>
    </types>
    <types>
        <members>*</members>
        <name>FlowDefinition</name>
    </types>
    <version>36.0</version>
</Package>

works, but a retrieve using:

<?xml version="1.0" encoding="UTF-8"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
    <types>
        <members>Issue280</members>
        <name>Flow</name>
    </types>
    <types>
        <members>Issue280</members>
        <name>FlowDefinition</name>
    </types>
    <version>36.0</version>
</Package>

only retrieves the flow definition and not the flow. If you want to be specific, you have to use the name Issue280-1 for the Flow member name.

Illuminated Cloud is, of course, specifying member names as reported by the API itself. Let me research a bit and see if I can figure out whether this is a bug in Salesforce or something API clients are supposed to understand and compensate for. I can certainly add a workaround where I add a member with a -1 suffix on it for that metadata type, but something tells me I'm not supposed to have to do that.

[Scott Wells]

Okay, I just reported all of this to my contacts at Salesforce. I'm willing to bet that the behavior I detailed above is also the root cause of the linked issue above. I bet that it's using the same reported API name for the Flow that it's using for the FlowDefinition when performing a bulk retrieval.

Let me see what kind of response I get from Salesforce on this. I'd prefer not to put a workaround in if it's going to be VERY short-lived. In the interim, you can work around this by using a subscription type of Package.xml instead of Selected and specifying either * or <flowName>-1 for the member name in the Flow type. If you don't have a package.xml file already, you can get one easily by performing a full metadata retrieve which includes one for the requested metadata.

[Scott Wells]

Hmmmmm...it appears that workaround doesn't...work...around. I'm probably going to have to add some temporary internal workaround for this that knows about the bug and accommodates for it. I'll see if I can include something in the next release that addresses this for you.

[Andrew Clunis]

That -1 is actually the version number of the flow! Each time you want to change a flow you actually need to publish a new version of it (at least once it has been activated at least once). Salesforce is slowly glimpsing that there is a need for version tracking of metadata, but, unfortunately, their one-off implementation of it for Process/Flows pretty badly breaks the workflow of folks using external repositories and deploy tools.

[Scott Wells]

That helps quite a bit, Andrew. The underlying bug is still the same, but at least there's a deterministic way to know the valid metadata names for Flows based on the version numbers from the corresponding FlowDefinitions. Should be much easier to accommodate.

[Scott Wells]

Okay, I revisited the workaround above and it does work if you specify * for the metadata type Flow in your package.xml. I ended up revisiting that after looking at what it would take to put a workaround into the plugin itself and it wasn't very pretty. Andrew, please let me know if you can manage these flows using that workaround or not.

[Scott Wells]

FYI, 1.7.3.8 includes an actual workaround for this issue. For more information, see this thread:

https://groups.google.com/a/illuminatedcloud.com/forum/#!msg/qanda/v89IKSOYyQc/FF7CTAhXAgAJ