Add support for force:source:deploy using --manifest option

[Tom Fuda]

We are currently living in a world where we’re using a mix of scratch orgs for continuous integration and test automation, and Developer Edition orgs for development. We use force:source:push to push code into our scratch orgs, based on the project structure defined in sfdx-project.json. We also use force:source:deploy to deploy our code into our DE orgs. We’re finding that using the “sfdx force:source:deploy --sourcepath force-app” option doesn’t give use the fine grained ability to deploy exactly what we need. Therefore, we created a classic package.xml manifest file to give us this control. We’re able to deploy using “sfdx force:source:deploy --manifest manifest/package.xml” from the command line, however, it does not appear that IlluminatedCloud supports this deployment option: https://www.screencast.com/t/MVCpp4tpplH.

[Scott Wells]

Tom, you can deploy from a scratch org project to a non-scratch org, but it's not via the metadata subscription. Please look at this demonstration of the functionality:

https://youtu.be/Jt_BVrBOARM?t=240

I believe that should provide what you're wanting.

[Tom Fuda]

So the use case is that we have a large project. We know exactly what we want to deploy to the DE org, and it is a subset of what is in the force-app folder. We’ve created a manifest/package.xml file to describe exactly what we want to deploy in a repeatable fashion. The thing is, when I do a “Deploy All Metadata” and choose “Module ‘connection-name’” as the content, it looks like IC then crawls the force-app folder and creates a temporary package.xml (named something like module_name_connection_name-package.xml) and then executes “sfdx force:source:deploy --manifest manifest/module_name_connect_name-package.xml” (using the temporary manifest, instead of the predefined one that I’ve already got in the manifest folder). The funny thing is that when I look at the log file, I can see that IC is actually detecting the presence of my manifest file, but it ignores it, and instead uses the temporary one it just created. Alternatively, having to explicitly select the components I want to deploy via the Context, or Custom deployment options doesn’t really seem viable, given the number of components I’m talking about.

‌

Here’s a log excerpt showing that IC is detecting the presence of my package.xml:

2019-09-26 15:47:36,906 [5201794] INFO - r.ForceComSfdxMetadataDeployer - Package.xml file found at C:/Users/tom/Development/RubyMine/IlluminatedCloud/xxxxxxxxxxx/manifest/package.xml
2019-09-26 15:47:36,906 [5201794] DEBUG - r.ForceComSfdxMetadataDeployer - Attempting to create a temporary package.xml file for selective deployment.
2019-09-26 15:47:36,913 [5201801] DEBUG - r.ForceComSfdxMetadataDeployer - Created temporary package.xml file 'C:/Users/tom/Development/RubyMine/IlluminatedCloud/xxxxxxxxxx/manifest/xxxxxx_xxxxx_xxxxx_xxxxx-package.xml'.
2019-09-26 15:47:36,913 [5201801] DEBUG - atedcloud.util.CommandLineUtil - Running command line: 'C:/Program Files/nodejs/sfdx.cmd force:source:deploy -u xxxxxxxxxx -x C:/Users/tom/Development/RubyMine/IlluminatedCloud/xxxxxxxxx/manifest/xxxxxx_xxxxx_xxxxx_xxxxx-package.xml --json' from working directory: 'C:\Users\tom\Development\RubyMine\IlluminatedCloud\xxxxxxxxx'.

‌

‌

‌

[Scott Wells]

Ah, I see. That makes sense. Let me chew on it. It's definitely doable...I just want to think about how to implement it so that it fits well with the current functionality and doesn't seem like a bolt-on beside it.

[Tom Fuda]

So, I discovered what might be considered a bug, or, perhaps an “undocumented feature”. As stated earlier, I had created a folder called “manifest” and a file within that folder called “package.xml”. Then, I tried to do the “Deploy All Metadata” operation and observed that IC creates it’s own temporary manifest file with a different name. However, I then discovered that when I closed this project, and then re-opened it, I got the error indicating a module is required. I clicked the “Resolve” link in the alert dialog, and this resulted in the following: https://www.screencast.com/t/PPjb7dP7f. I didn’t do anything other than click “Resolve”, and IC seemed to automagically choose manifest/package.xml. And when I view the IC subscription configuration, I now get this: https://www.screencast.com/t/dbRvQG2Uc. As you can see it has now selected my package.xml as the subscription, and I cannot change it. The radio options and file chooser are disabled. Strange. So, basically my repro case for this is:

‌

• Using an existing SFDX project with source format metadata, create a manifest folder and package.xml in the manifest folder.
• Connect to a DE org and do a “Deploy All Metadata” operation. Observe that IC creates temporary manifest in the manifest directory (with name other than package.xml). It will successfully deploy the metadata (assuming no deployment errors).
• Close the IDE and re-open. Observe the error about a module being required.
• Click “Resolve”. Observe the dialog indicating that manifest/package.xml has automatically been selected.
• Open the Tools > Illuminated Cloud > Configure Module… and view the Illuminated Cloud / General tabs on the module configuration dialog. Observe that Package.xml has been selected as the Subscription, and that you cannot change it.

‌

[Tom Fuda]

More info; now when I do the “Deploy All Metadata” operation, it still creates the temporary manifest, but the temporary manifest now effectively matches what is specified by my package.xml (with the exception that “wildcards” are expanded to explicitly list every artifact). For example, my package.xml specifies <members>*</members> for the ApexClass type, but the temporary manifest explicitly lists every Apex class contained in force-app/main/default/classes (which is fine). So, it actually seems to be doing what I want it to do

[Scott Wells]

Yes, if you do a Deploy All Metadata for the entire scratch org, IC generates a temporary subscription based on the local metadata it finds that's also selected for deployment (which could be the entire project). So it sounds like it's working okay for you?

[Tom Fuda]

Yes, it looks like it is working as I would like as far as deployments go. I guess the only thing that is a bit odd is that I could not explicitly select manifest/package.xml as my subscription. It only happened after I closed / reopened the project and resolved the error about a module being required. And I cannot change the subscription now if I wanted to. The radio options are all disabled.

[Scott Wells]

I'll keep this open and take a look at the original workflow that we discussed...allowing you to select a package.xml file as a filter for ad hoc deployments from scratch orgs to non-scratch orgs and retrievals from non-scratch orgs into scratch orgs. I think that makes good sense, and it makes it much more explicit.

[cognivator]

Is the the possibility of selecting a custom manifest file still on the table?

I ask because it would be useful to have several package.xml files (by different names, in the /manifest folder, say) for retrieving and deploying different parts of our large app, depending on the project / task at hand.

Awesome functionality would allow setting the package.xml to use while kicking off the retrieve / deploy operation, and have that be the remembered option going forward.

Less awesome would be to allow override each time, but without remembering the previous selection.

Minimum is allowing a different package.xml selection from IC configuration - instead of pinning it to /manifest/package.xml with no change allowed (which is current functionality for a DX OAuth connection.)

Note: For what it’s worth, this is standard ops in VSCode… the free alternative. I like JetBrains / IC better on most other levels, but this is a thorn.

[Scott Wells]

Issue #1681 was marked as a duplicate of this issue.

[Scott Wells]

Pretty sure this was delivered quite a bit back. Let me know if not.