Compare With Server behaviour in deployment dialogue seems to be partially erratic

[René Görgens]

This is a mixture of question, proposal and potential minor bug.

I’m deploying the context via Metadata API from local source/ configured connection (org A) to another connection (org B).

I’m seeing:

• What’s only local (green bold) – org A only
• What’s local+server (blue bold) – orgs A ∧ B

However I don’t see (whether I select Deploy All Metadata or Deploy Modified Metadata) which metadata in org B is unchanged from org A, unless I do a comparison (Compare With Server).

• Notes on the comparison:

  • I’m not sure it’s correctly scoped. If I select 1 metadata (e.g. 1 Apex class) and click the compare icon, then I see the comparison as expected for this particular class. However if I select 2 (out of several) Apex classes and click the comparison icon, then a) all local Apex classes are compared with all remote classes b) but the diff is completely empty.
  • Trying to compare individual subscribed children results in a comparison with a file in which the full scope of the object’s subscriptions are presented on the local side. Maybe this just takes some getting used to. A full comparison can be achieved by selecting the parent object for comparison, then I’m given the possibility to compare all subscribed children against the target (which is logical).

My main point is that there doesn’t seem to be an Eclipse like comparison between local and remote upon deployment, where as far as I recall (I last worked with Eclipse Kepler), in Eclipse non-modified metadata were grayed out. You could overwrite them but there is hardly a point in doing so.

In the end, this is a strong case for using version control, but I suggest:

• Fixing the problem with the comparison icon (if there is one)
• Reflecting on whether an auto-comparison could be done for Metadata API source format and unmodified metadata could be detected/ grayed out

‌

[Scott Wells]

I’m not 100% sure I follow. The Compare with Server action in the build options dialog operates on the selected items in the metadata tree that if all items are known to be present both locally in the project source files and on the server. And “known” in this case is based on the latest cached version of the server’s metadata list, so if that’s stale it would need to be refreshed. When clicked, it will retrieve exactly those metadata objects from the server and open the diff tool (either the IDE’s default or an external tool if so configured) with the server version on the left and the local/project version on the right.

This is definitely not meant to be a replacement for version control. It’s a way to spot check things you’re about to deploy/retrieve/delete so you can know for sure what will actually happen. There’s a broader-scale version of this action in the main IDE that operates upon the selection there and isn’t constrained by whether there is known to be a server counterpart to determine whether it’s enabled or not. If invoked on the entire project, it does effectively give you a full overview of the local project metadata vs. the org metadata. You can take that even further by using Retrieve Metadata > Retrieve for Merge configuring the things you wish to include and can get a very solid feel for what’s present only in the org, what’s present only locally, and of the things that are present in both places, what’s different. If you’re going to do that often (and I do it VERY often), I’d recommend configuring the IDE with a world-class external diff tool such as Beyond Compare or Araxis Merge. I use the former and it makes comparing the org and project a very pleasant experience when coupled with IC2’s Compare with Server / Retrieve for Merge actions.

Perhaps this is a misunderstanding of the purpose and (intended) behavior of that action? If you’re not seeing it behave as described above, please provide concrete steps to reproduce the errant behavior.

[René Görgens]

Thank you Scott, I’ll try to reproduce it and provide a screenshot.

[René Görgens]

A number of Apex classes are deployed both to org A (source files) and org B. These classes are identical due to recent deployment.

Selecting 1 class and invoking Compare With Server:

Selecting 2 classes and invoking Compare With Server:

Instead of the selected 2 classed, the 6 classes I didn’t select are presented, but their diff isn’t retrieved:

‌

So the action operates on the inverse scope (not on the full scope as mentioned initially) and the diff retrieval for the inverse scope fails/hangs.

[Scott Wells]

Ah. Let me provide a little context on how the JetBrains directory diff tool works. In your example it’s showing the directory of retrieved classes on the left and the project’s directory of classes on the right. The toolbar is configured to display things that exist only on the left, things that are different between left and right, and things that exist only on the right. All of the green files on the right are present because of the last toolbar option. Disable the green left arrow and it’ll show only things that are exist on the left (i.e., org-only files) and things that are different. For example, this:

becomes this:

Unfortunately it’s showing those three files as different even though the only differences are in line separators. Note that I have configured the file diff panel to ignore whitespace and empty lines and to show a context diff, so that’s certainly better.

But that’s where an external diff tool like Beyond Compare comes in. Here’s the resulting diff when I use that as an external diff tool:

Unfiltered - showing differences with no filter for “unimportant” differences

Filtered - showing only things that are newer on the left and not showing “unimportant” (i.e., minor) differences

You get a VERY useful view showing what you really care about in terms of differences between org and local. Not trying to sell you on yet-another-commercial-tool (Beyond Compare is a commercial product), but while the default JetBrains diff tools are decent, dedicated tools in that space are worth the price of admission IMO. I’ve been a licensed user of Beyond Compare for…15-20 years, I think?

‌

[Scott Wells]

You might take a look at JetBrains' documentation on their Diff & Merge tools/features for a deeper understanding of how you can configure them for the best signal-to-noise ratio:

https://www.jetbrains.com/help/idea/settings-tools-diff-and-merge.html
https://www.jetbrains.com/help/idea/comparing-files-and-folders.html
https://www.jetbrains.com/help/idea/differences-viewer.html

[René Görgens]

Thanks a lot for the guidance and tips :) I downloaded Beyond Compare and actually purchased a license. I see that it can do way more than just compare text.

My experience with the JetBrains diff tool is small, indeed I forgot to enable Show Equal Files:

‌

I assume that launching the comparison on all items in a folder once 2 or more are selected is an IDE default which cannot be changed. So working as expected then.

[René Görgens]

Working as expected

[Scott Wells]

Let me know if you want some tips on how best to configure Beyond Compare to work with IC2. There are a few small tweaks that really improve its default behavior.

And yes, unfortunately there’s nothing I can do to alter the behavior of the default diff tool, at least beyond what I’m already doing. I actually did chat with JetBrains about some ideas a few years ago, but unfortunately nothing really came of it other than some interesting discussion.

[René Görgens]

I’d love to have your advice. Here’s my current config:

‌

[Scott Wells]

Those settings should be fine. You’ll want to change some settings in Beyond Compare itself, though. I’ll attach an export of my session and file format settings that include my default behavior for a Folder Compare session and a file format for Apex. The Folder Compare session automatically starts a content-aware diff upon being started, saving an obvious step you’d perform every time. The Apex file format is already set up for the standard extensions for Apex source files and for Apex’s case-insensitive nature. You can import those using Tools>Import Settings. Then I recommend that you also set up both your Folder Compare view as:

‌

and your File Compare view as:

‌

You’ll likely toggle Minor off in one and/or the other view when you explicitly care about changes in whitespace and comments, but most of the time you’ll want that enabled so that you’re looking at true code/logic differences.

[Scott Wells]

Here are the exported Beyond Compare settings. Again, this only contains the Folder Compare session and Apex file type, so it should be safe to import directly without messing with your other config.

[René Görgens]

Much appreciated Scott, thank you :) I imported the settings

That’s quite a bit of work that went into defining the Apex Source file type! Very useful

‌

[Scott Wells]

Oh, I’d love to say that it was, but all I did was copy the Java file format and change the extensions/case-sensitivity. It doesn’t even include the Apex keywords that aren’t Java keywords. But it works well in spite of that.

[René Görgens]

I did expand on this today and updated the Apex Source file type:

• I got the list of Apex keywords which is substantially longer than Java keywords. I wanted to be able to style SOQL differently
• I added Apex annotations

‌

I ran into some quirks with the BC4 regex engine, otherwise I would have distinguished between method declarations and method calls, as Idea does. I’ll raise a ticket with BC support concerning lookbehind and lookahead support, which I found to be missing (they say that a PCRE subset is supported, but suggest that ? can be used whereas I found it inadmissible).

My purpose was to align BC’s display with Idea’s Darcula theme to have a mostly seamless experience when invoking BC.

You’ll find the BC settings for the Apex file type and my working file attached.

[René Görgens]

Beyond Compare settings for Apex Source file type

[René Görgens]

Apex keywords working file

[Scott Wells]

Very nice. Thanks!

[Scott Wells]

FYI, it’s going to be virtually impossible to distinguish when certain “reserved” words are used actual keywords and when they’re not, e.g., get is used for property read accessors but is also a valid method name, e.g., System.Map.get; set is used for property write accessors and is also a system data type name; Case is an SObject type name and a control flow keyword. Similar for keywords that are present in both the Apex and SOQL/SOSL grammars. Without a recursive descent parser (or similar), there’s insufficient context to know how those keywords are being used. This has been one of the very fun things about writing and maintaining an Apex/SOQL/SOSL parser for IC2 over the years.

[René Görgens]

I can well imagine :)

I do think that the omission of get and without in https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_reserved_words.htm is inconsistent, as it includes set and with.

Here’s my bug report to Scooter Software:

Date: Thu, 20 Oct 2022 at 22:35
Subject: Bug report: Modifiers not allowed in regular expression

Dear Support,

I recently purchased Beyond Compare, and it's amazing :)

However I ran into a regex issue while trying to configure a new file type.

• Context: I'm working on an Apex Source file type (which started as a clone of the Java Source file type)
• I'm trying to isolate methods, mostly method calls i.e. the text between a dot and an opening parenthesis. Example:

Ingredient__c ingredient = TestFactoryWoe.sampleIngredient(dish.Id, item.Id);

For this purpose, I've written a regex that uses positive lookahead: ([\w]+)(?=\()

So as to avoid to highlight the opening parenthesis, while using it to detect the method.

However when I try to save this regex, I get the message: Invalid regular expression: Character expected at position 9

‌

The BC help states:

• The regular expression support in Beyond Compare is a subset of the Perl Compatible Regular Expression (PCRE) syntax.

But also:

• Modifiers allow changes to the matching behavior from that point on. [...]  Use (?i) and (?-i) to control the case sensitivity of matching.

I find that any attempt to use a question mark (modifier) anywhere in my regex results in Invalid regular expression

Is that a defect or working as expected?

Technically, this prevents me from using negative lookbehind and positive lookahead, for example.

The applied workaround obliges me to highlight more text than I'd want to.

Thanks a lot in advance.

[Scott Wells]

Yeah, I updated my Apex file format to include all Apex/SOQL/SOSL keywords recognized by my Apex lexer. They’re all glommed together, and keywords that are also valid as identifier names will result in keyword highlighting of those identifiers, e.g., case and Case, but it now covers all keywords on those languages' grammars.