Multiple errors and ui lag when .sfdx directory does not exist

[John De Santiago]

I originally came across this issue when setting up a new Macbook Pro M1 14” for development. However, I was able to narrow down the issue and reproduce on a Macbook Pro M1 16”. As best as I understand even though I had installed the CLI and run some commands I hadn’t connected to any orgs which meant the “.sfdx” directory never got created. I downloaded an existing project from a git repo and opened that project in both VSCode and in IntelliJ. The errors started and the UI became unresponsive. After a few attempts to reload the project I eventually manually created the “.sfdx” directory and the project loaded without errors.

Error Message:

Failed to obtain the org list from the Salesforce CLI: • status = 1 • message = ENOENT: no such file or directory, scandir '/Users/john/.sfdx'

‌

Steps to reproduce:

• Start with a fresh install of SF CLI and the local “.sfdx” directory has not been created, or delete the “.sfdx” directory

• Path 1

  • Create a new SFDX Project
  • Navigate to Tools > Illuminated Cloud > Configure Application > Connections
  • Click the “Refresh CLI Connections” button
  • The error should appear, but UI will remain responsive after a few messages show up.

• Path 2

  • Open an existing SFDX Project that was previously configured with an sfdx connection.
  • Errors should start appearing on start-up of the project

  • If errors do not appear, then…

    • Navigate to Tools > Illuminated Cloud > Configure Application > Connections
    • Click the “Refresh CLI Connections” button
    • Errors should appear

‌

[Scott Wells]

John, I think this may be a side-effect of the same underlying issue as #2385 which is itself due to an issue with the CLI’s logger. Salesforce (specifically Shane M.) is working on a proper resolution to that whole class of issues, so I’ll likely sit tight on this until they release their fix and then we can see if it just magically goes away…

[Scott Wells]

John, one question about this. Are you talking about the .sfdx directory under the project itself or the one under the user home directory? I’m guessing the latter as that’s where connection information is stored, and your steps-to-reproduce are specific about refreshing the connection list.

[John De Santiago]

Under the home directory. No issues with the .sfdx folder in the project being present or not.

[Scott Wells]

Either way I don’t see those errors. When I delete/rename the .sfdx and .sf directories under the project and refresh connections, it all works fine because that’s not where it’s picking up that information. When I do the same thing for those directories under the user home directory and refresh connections, the CLI creates those directories and the connection list is empty (or rather only contains IC2-native connections).

That further supports my assertion that what you’re seeing is likely a side-effect of the CLI issue linked above, so I guess we’ll see if the problem continues to occur after Salesforce releases a fix for that. I’ll put this on pause until then.

[John De Santiago]

I agree it is likely the same issue that you mentioned, with it being a CLI issue. As an additional test I deleted my .sfdx home directory folder and in a terminal window ran sfdx org list, and the result I get is the same ENOENT error.

‌

[Scott Wells]

Can you paste the full error you see for that command, ideally with the command run with the --json flag which is how IC2 runs it? That will help me to correlate what you’re seeing here with the linked issue.

[John De Santiago]

sfdx org list --json
{
  "code": 1,
  "context": "OrgListCommand",
  "commandName": "OrgListCommand",
  "message": "ENOENT: no such file or directory, scandir '/Users/john/.sfdx'",
  "name": "Error",
  "status": 1,
  "stack": "Error: ENOENT: no such file or directory, scandir '/Users/john/.sfdx'",
  "exitCode": 1,
  "warnings": []
}

‌

[Scott Wells]

Thanks. That’s not quite the same as what we’re seeing in the other. In the other, at least part of the issue is that the error message is emitted to stdout alongside the structured JSON response, but it’s not part of that response, so the overall response is malformed. In what you’ve provided, the JSON response is properly structured, though I’m confused as to why the CLI isn’t creating that directory on-demand. Let me check with Shane.

[John De Santiago]

Just in case, here is my CLI version details

sfdx-cli/7.204.6 darwin-arm64 node-v18.15.0

‌

[Scott Wells]

Pinged Shane, but for reference, I can’t reproduce that locally:

$ mv ~/.sfdx ~/_sfdx
$ sfdx org list
No non-scratch orgs found.
No active scratch orgs found. Specify --all to see all scratch orgs.
$ sfdx org list --json
{
  "status": 0,
  "result": {
    "nonScratchOrgs": [],
    "scratchOrgs": []
  },
  "warnings": []
}
$ ls ~/.sfdx
key.json*

So I first deleted (or in this case, moved) ~/.sfdx out of the way, then ran the org list command. Based on what you’re saying, that should fail with ENOENT, but as you can see above, for me the CLI stubs that directory and provides a valid response. So obviously there’s some significant difference in behavior that you’re seeing. I wonder if a scorched earth reinstall of the CLI might resolve that for you. It sure seems to resolve a large swath of bizarre CLI issues in my experience (both my own and based on helping others).

[John De Santiago]

Interesting. I also tried a move command to rename the directory and was able to reproduce the issue. Would more details about my machine or environment help?

Also, I did do a scorched earth reinstall thinking that might help but I got the same results.

[Scott Wells]

Okay. I’d recommend you uninstall the CLI completely as follows:

https://developer.salesforce.com/docs/atlas.en-us.sfdx_setup.meta/sfdx_setup/sfdx_setup_uninstall.htm

then reinstall and see if the same issue is still present. No kidding, that seems to clear up 75-80% if the “weird” issues I and others see with the CLI. If it doesn’t, I’ll likely send you to the CLI public issue tracker since you’re seeing different and likely unexpected behavior.

[Scott Wells]

If you’ve done a true scorched earth reinstall as listed above--removing all of those cache directories and everything--and the same issue still occurs, you should log a bug here:

https://github.com/forcedotcom/cli/issues

Include the exact steps to reproduce the issue with an expected result as I’ve shown above where it silently stubs the ~/.sfdx directory and returns an empty org list.

[John De Santiago]

Will do. Interestingly, after a fresh install, I did notice that when I run a command, it does auto-create a .sf directory but not a .sfdx directory.

‌

[Scott Wells]

Okay, I just reproduced this on my Mac M1:

$ mv ~/.sfdx ~/_sfdx
$ mv ~/.sf ~/_sf
$ sfdx org list --json
{
  "code": 1,
  "context": "OrgListCommand",
  "commandName": "OrgListCommand",
  "message": "ENOENT: no such file or directory, scandir '/Users/swells/.sfdx'",
  "name": "Error",
  "status": 1,
  "stack": "Error: ENOENT: no such file or directory, scandir '/Users/swells/.sfdx'",
  "exitCode": 1,
  "warnings": []
}

Fine on Ubuntu Linux:

$ mv ~/.sfdx ~/_sfdx
$ mv ~/.sf ~/_sf
$ sfdx org list --json
{
  "status": 0,
  "result": {
    "nonScratchOrgs": [],
    "scratchOrgs": []
  },
  "warnings": []
}

Fine on Windows:

$ mv ~/.sfdx ~/_sfdx
$ mv ~/.sf ~/_sf
$ sfdx org list --json
{
  "status": 0,
  "result": {
    "nonScratchOrgs": [],
    "scratchOrgs": []
  },
  "warnings": []
}

‌

[Scott Wells]

Filed:
https://github.com/forcedotcom/cli/issues/2222

[Scott Wells]

John, not sure if you saw, but I have it pretty nailed down now. There’s a confluence of things happening here, and the log rotation issue is kind of part of it as it was included in the reproduction when ~/.sfdx was missing but ~/.sf was present. However, when both are missing, Windows and Linux behave properly while Mac fails exactly as you’re seeing. I’ve updated the linked Salesforce CLI issue with all of those details. Looks like this is something for them to address. Strange that it would be Mac-specific. If nothing else, I would at least expect Linux to behave in a similar manner.

[Scott Wells]

John, it looks like this should be resolved in today's CLI update:

https://github.com/forcedotcom/cli/issues/2222#issuecomment-1593559073

I've verified the fix on my Mac. Can you update and confirm that it's fixed for you as well?

Also, I'm resolving this here since it was a CLI issue. Admittedly IC2 could have handled the situation better, but then again, the situation should never have occurred.