Listing connections extremely slow

Issue #1069 resolved
nivgo created an issue

When I'm opening IntellIj or switching for example to the SOQL Query tab, illuminated is listing my connections again, I have a lot of connections including DX connections and the process is taking sometime around 10 minutes.

Can you support some sort of lazy loading like loading only the connections that my project modules are assigned to or maybe on demand loading by the connections picklist current selection?

Comments (9)

  1. Scott Wells repo owner

    Niv, IC does cache the SFDX portion of the connection list after the first load until it sees that those details have changed by monitoring the last modified timestamps of the corresponding files under your .sfdx directory. I'm more concerned that it's taking upwards of 10 minutes to list your connections. How long does it take to run sfdx force:org:list from the command-line? That's all that IC is doing. If that's taking a long time, we need to follow up with the SFDX team and figure that out. Even with perfect optimization in IC, it's going to need to consult the CLI for a list of orgs and for org details, so if that's excruciatingly slow then IC is going to suffer as well.

  2. nivgo reporter

    The issue is that this is happening when I'm not on a DX project, actually most of my DX connections are not active and the majority of the connections I have are regular connections (not related to DX).

    Running "sfdx force:org:list" takes around 30 seconds.

  3. Scott Wells repo owner

    Ah, okay. So there are a few things here...first, it sounds like when you're in a non-SFDX project, you're okay not seeing the SFDX scratch orgs in things like the Anon Apex, SOQL Query, and Log Viewer tool windows, correct? That's why it lists them when you go to those tools...so that they can be available for selection. I've considered having a configurable option to say whether to show SFDX scratch org connections in non-SFDX projects. Sounds like an option like that would address this for you.

    I'm also curious as to why sfdx force:org:list takes 30 seconds from the command-line but upwards of 10 minutes from within IC. It would be good to get some debug info on that so I can see where that time is going. You can enable debug logging for Salesforce DX as documented here:

    http://www.illuminatedcloud.com/support/debuglogging

    and your idea.log will include the details of every interaction with the sfdx CLI. Do you mind enabling that and reproducing the issue, then posting your idea.log here for review (or email it if you'd prefer)?

  4. Rohan Gore

    @RoseSilverSoftware It would be good to have a switch for when/where and when/where not to list scratch orgs as they take time in general to show up.

  5. Scott Wells repo owner

    Yes, I'm thinking about adding, at a minimum, a flag to denote whether to include scratch orgs in the connection listing when working in a non-SFDX project. I would also likely allow you to override that setting when choosing a connection for Anonymous Apex, SOQL Query, and Log Viewer if you'd like, perhaps with an additional <Include Scratch Orgs> entry at the bottom of the standard connection list.

    Unfortunately this gets a little more complicated with one of the upcoming enhancements...I'm planning to use the SFDX CLI to add support for OAuth since it already manages refreshes and such. That would mean that if you wanted to use OAuth connections, you'd still be stuck with the overhead of the SFDX CLI for listing connections. I'll need to think about that a bit more since they're obviously competing goals.

  6. Scott Wells repo owner

    Hi, Niv and Rohan. I just had an opportunity to speak with some of the folks on the SFDX CLI team while at Dreamforce and they recommended that you guys run:

    sfdx force:org:list --clean
    

    which will remove entries for stale/expired orgs and keep them from being validated when further force:org:list commands are run including those from Illuminated Cloud. Would you mind trying that to see if it helps?

  7. Scott Wells repo owner

    Good to hear. Still not ideal, but hopefully that'll make the experience a bit better until I can provide a true fix in IC. The SFDX CLI team also knows that there are major opportunities for performance improvements in the CLI commands that are making this run long, so once those are implemented, I'll revisit any intermediate workarounds for this problem. I'll keep you posted as I work through each phase.

  8. Log in to comment