Synchronize project/org for non-source-tracked orgs

[Mike Mikula]

Generalizing this as a request for an ability to synchronize metadata between a project and the associated org when source-tracking metadata is not available. I have some extensive thoughts now on how this can be done (hopefully) reasonably efficiently and may try to tackle it shortly.

ORIGINAL DESCRIPTION

As a user I would like to select the source folder and only deploy the diff between my IDE and the server.

[Scott Wells]

Hi, Mike. Are you finding that the Deploy Modified Metadata action doesn't work properly for you? That action compares the last modified timestamps of your local files to the last modified timestamps of the corresponding metadata and only deploys things that are newer (updated or created) in the local project. It doesn't, however, try to determine metadata which should be removed from the server. That must occur as a separate action, though IC does have some nice facilities for isolation orphaned/abandoned metadata in the server. If that action isn't working properly for you, can you explain the shortcomings?

Or are you perhaps asking for something different such as a true source tracking-like mechanism similar to Salesforce DX's push/pull model?

[Mike Mikula]

Thats correct, I’m looking to have a similar mechanism to the sfdx source tracking. Sometimes files are updated/ touched/ reverted serverside that really should be getting updated based on my local environment. Having the option to do a more comprehensive check I think would be nice for ensuring my IDE/source control wins.

[Mike Mikula]

The other issue that I just thought about is if I’m locally changing between branches in version control. It may incorrectly make it seem like certain files are more updated than others.

[Justin Julicher]

I agree with Mike’s comment - is it possible to use the Git last modified when the state of the file is unmodified and the modified date if it is?

you would need to do for each file something like:

git log -1 --pretty="format:%ci" /path/to/repo/anyfile.any)

‌

‌

[Jacob Mather]

Now that Salesforce has Source Tracking in Sandboxes it might be worth having an override option for “Sandbox” connections to use sfdx push/pull, similar to how you can opt to “Use push/pull instead of deploy/retrieve/delete” for scratch orgs.

[Scott Wells]

Hi, Jacob. That's been supported since source-tracked sandboxes were in pre-release. Just check Use push/pull instead of deploy/retrieve/delete on the sandbox connection and it'll be treated as if it were a scratch org. Similarly you can check Use deploy/retrieve/delete instead of push/pull on any scratch org connection and it will be treated by IC2 as if it were as non-scratch org. I actually do that pretty much all the time as I prefer the more fine-grained deploy/retrieve/delete operations to push/pull.

You can find more information in the user guide here:

https://bitbucket.org/RoseSilverSoftware/illuminatedcloud/wiki/User_Guide/Configuring_Connections

[Mike Mikula]

one issue I’ve found with retrieve is that it resets SFDX tracking. The only way I have gotten around this is to do a mdapi retrieve to a folder outside of force app then copy files over.

retrieves are nice and sometimes still necessary as SFDX has some blank spots with what it actually tracks. I had to build a script for profiles that dynamically creates all the related packages so you can get a complete profile. Interesting how much of a burden that can be.

[Scott Wells]

Yes, using raw metadata operations against a source-tracked org can wreak havoc on the source tracking information. Theoretically the next push/pull should catch things up properly--and it often does--but it also often does not. There's a reason that Salesforce added explicit force:source:tracking:clear/reset commands to the CLI. Better that than trying to clean things up manually by whacking things under .sfdx and/or the source tracking server records.

I've also found that a raw retrieve can produce quite different results for the same metadata than force:source:pull. After over a decade of doing the former and learning many of the quirks, tips, tricks, etc., I've also learned that I prefer having fine-grained control over how metadata is moved back and forth between the local project and the server vs. using source tracking. That's not to disparage the work that Salesforce has done as it's night-and-day better now than the early days, but I just prefer to have things more under my own control.

[Mike Mikula]

I do however prefer push pull as it keeps my org in sync with destructive changes other team members make.

[Scott Wells]

Yeah, that's a fair point. It does attempt to be a true synchronization mechanism. I'm still considering adding such a concept to IC2 that goes beyond Deploy Modified Metadata and also takes into account things that it has deployed previously but that no longer exist locally, offering to remove those from the server if present. Similar the other direction where if you have something locally that's been deployed previously but is no longer on the server, offer to remove the local copy. But if I were to do that, I would show the exact changes that would be applied in both places with the opportunity for the user to opt out of any recommended changes that are incorrect, potentially omitting them from future sync attempts. But that's not on my near-term list...just something I've considered as a logical extension to IC2's existing raw metadata deploy/retrieve/delete operations.

[Mike Mikula]

Understandable

I find it challenging to not use SFDX with our current development process. We have developer org sandboxes with SFDX beta feature enabled. With many other engineers making changes in their respective orgs that may be destructive it can be tough to keep my org up to date. For example some times master may have changes that are destructive that haven’t yet been deploy to production. If I have SFDX tracking enabled as I cut a new branch off master its fine as it handles it. If I reset tracking because I did a retrieve and master != production, I would have to:

• Refresh my org from production
• Reset my branch to a point in master that was last deployed to production
• Push to re-establish tracking
• Reset my branch to head of master
• Push all destructive changes

Definitely a bit tedious including these other issues with SFDX:

• Changing a picklist to global vise versa
• Adding record type page layouts
• Profiles changes
• Permission set changes depending on level being edited
• Pushing a profile/permission set that has a feature metadata only available to production org! (This one really annoys me because we can’t track it so I had to build a clean script if I pull from prod)

‌

[Scott Wells]

Issue #2091 was marked as a duplicate of this issue.