Metadata pulled into incorrect directory when multiple source roots

[Eric Alexander]

Steps to reproduce:

​

​1. Create new project

​2. Connect to new scratch org.

​3. Finish project setup

​4. Add new directory (otherOrgMetadata) under force-app

​5. Make that new directory a source root

​6. Ensure default root is the force-app/main/default

​

1. ​Then right click on the new directory

​2. IlluminatedCloud -> Retrieve Metadata

​3. Change from context to custom

​4. Select a connection for another org

​5. Pick anything to pull

​

​Click button to retrieve.

​

​End result -> pulled into force-app/main/default/xxxx

​Expected -> pulled into force-app/otherOrgMetadata/xxx

[Scott Wells]

Well crap...I just typed up this really awesome (and verbose) description of my attempt to reproduce this...and lost it. Rather than type it again, I'll summarize by saying that I'm able to reproduce the behavior where you perform a contextual retrieval on a non-default source root/package directory and the retrieved metadata is still placed in the default source root/package directory. It makes sense that you'd want those retrieved files remapped to the contextually-selected source root, though. Can you please confirm that to be the crux of this issue from your perspective so I can make sure I haven't missed something else?

[Eric Alexander]

That is correct.

‌

The other issue is when they pulled metadata is only custom fields, if you try to move the metadata to the context directory it does not move properly. To reproduce pull a custom field, then try to drag and drop that metadata from default to context and observe the result.

[Scott Wells]

Thank for confirming. Regarding the field issue, what is the resulting directory structure? I ask because I did find an issue when I structured things exactly as described in your steps-to-reproduce (or at least as I understood them) where relative paths were being duplicated by the CLI during the retrieval, e.g.:

    {
      "createdDate": 0,
      "fileName": "C:\\Users\\Scott\\AppData\\Local\\JetBrains\\IntelliJIdea2022.2\\plugins-sandbox\\system\\tmp\\issue_2175-demo-retrieve\\tmp\\force-app\\main\\default\\main\\default\\triggers\\ExpenseController.trigger-meta.xml",
      "fullName": "ExpenseController",
      "lastModifiedDate": 0,
      "type": "ApexTrigger"
    }

Notice that it has main\\default\\main\\default\\triggers with main\\default repeated. I can elaborate on the config that leads to that behavior and how to restructure/reconfigure slightly to avoid it. It's all based on CLI behavior and not due to IC2, though obviously it manifests visually in IC2 as a result.

[Eric Alexander]

This is what it starts as

After dragging the objects.Opportunity.fields folder to the otherOrgData directory this is what it looks like

‌

‌

[Scott Wells]

Eric, can you uncheck Compact Middle Packages in the Project view settings and make sure that you're drag-n-dropping the correct folder? That setting can make it awkward (at best) to move folders in cases like this.

[Eric Alexander]

unchecking compact middle packages does seems to allow the directory to be moved properly as I am moving the entire Objects folder in that view

[Scott Wells]

Okay, good. So the real issue is that a contextual retrieve on a non-default source root/package directory doesn't retrieve into that directory, correct?

[Eric Alexander]

As far as IC goes yes. I guess the other issue is not an IC issue and it is not a big deal anyway if we get the main issue resolved

[Scott Wells]

Eric, as I started working through the changes needed for this, I ran into a bit of a disconnect. One integral aspect of this use case is the use of Custom as the value for the Contents drop-down instead of Context, and when you select that you lose the notion of any contextual selection. Yes, you could argue that that’s somewhat artificial, but so far the concept of “context” has been pretty consistently tied to that value for Contents.

Given that we’ve now discovered that you can successfully move retrieved files to another source root/package directory once retrieved, and given that that’s how the CLI behaves as well based on the default package directory setting in sfdx-project.json, I’m inclined to leave things working the way they do now. Otherwise it introduces a new behavior where a retrieval that wasn’t activated based on context ends up reacting to the context under which the retrieval was initiated. I fear that will lead to its own confusion.

[Eric Alexander]

Scott,

I would use “Context” if I could. The issue is that if I leave it on context then NO metadata is displayed as available for retrieval. Most likely because the directory has nothing in it. If you could somehow show the metadata list in the context situation that would work but I suspect that will break something else since in a context situation with an empty directory the context is no metadata….Chicken and egg situation I think…..

‌

What if you added an option in the retrieve dialog to change the directory (from the list of source directories), via a picklist option, where the metadata goes when the “Custom” option is chosen? that would be a good work around…

[Scott Wells]

Hah! Yeah, you’re getting more than a bit of stream of consciousness from me right now. My last entry sounds much more conclusive than it actually is. I have a branch right now with the preliminary changes for what needs to happen and am primarily mulling on the UX aspects. The underlying use case is valid and I’m going to address it. Just trying to sort through the options for how IC2 determines the correct source root to use and whether that’s implicit, explicit, or a mix. More to come in a bit once it’s less sketch and more actual design…

[Scott Wells]

Okay, here’s what I’m thinking on this topic. If you start a metadata retrieval in a project with multiple source roots/package directories and switch the Contents drop-down to Custom, a new drop-down field will be added to the dialog to specify which source root/package directory should be used for retrieved files that could not be matched to local project files. If the retrieval was started using a context that can be uniquely resolved to a single source root/package directory, that will be the initial selection. If not, e.g., it was started from the project root node, the configured default source root/package directory will be the initial selection. Unmatched retrieved files will then be reparented to the selected source root (if needed) before being shown in the diff tool (retrieve for merge) or copied into the project directory structure (retrieve).

Make sense? Anything it doesn’t address?

[Eric Alexander]

Sounds that that would result in the desired outcome.

[Scott Wells]

Okay, it’s implemented as described above:

Testing some corner-cases to make sure that nothing ends up broken, but things look very good so far. Barring unforeseen circumstances (such as more major issues from the Winter ‘23 preview that’s already rocked the boat this week) it should be included in Thursday’s build.

[Scott Wells]

Delivered in 2.2.3.4.