Issues creating user trace flags for orgs with large numbers of users

[Scott Wells]

There's more on this topic here:

https://groups.google.com/a/illuminatedcloud.com/g/qanda/c/gommbKG3Ps8/m/zZMMPV1ICAAJ

but basically in orgs with very large numbers of users--typically portal/communities users--the query to enumerate those users can take a VERY long time.

There are several ways to address this. I'm going to use this issue to try a few and give the affected users an opportunity to provide early feedback.

[Scott Wells]

Here's a build that removes the ORDER BY clause from the User query and instead orders the result set locally. Let's see whether it makes a material difference. I'm concerned that just marshalling/transferring/unmarshalling the result set may be a non-trivial part of the time. This should tell us whether this simple change is a useful one, though.

To install it, download the archive (but don't extract it) and use Settings / Preferences > Plugins > Install plugin from disk (under the gear drop-down menu).

I'll be curious to hear your findings.

[Kris Goncalves]

Tested out the build. Compared to the previous behavior where I got constant freezes and inability for it to actually finish (for me and this specific org, at least)

Improvements

• IDE did not freeze - able to click and move window (Loading users modal) around
• Able to press cancel and abort successfully
• It did eventually complete

Cons

• It took 21 minutes to successfully load the users
• If I click “cancel” I got a unhandled exception: com.intellij.openapi.progress.ProcessCanceledException

So some progress! I did notice I have a lot of weird community users names (some with no letters or numbers - just a period, some with numbers) so I wonder if that makes it worse for my experience as well.

The goods news is my dev sandboxes worked really smoothly whereas before they were hit or miss still (even with just 5500 total users). The one I used for above has 4.8 million total users, 63,000 of which are active.

‌

[Scott Wells]

Hah! Okay...21 minutes isn't acceptable. However, filtering by IsActive = TRUE is worth trying, though it may just result in the query executing very slowly again. Hopefully not as Name is actually kind of a compound column whereas IsActive should be indexed given that most of the time you just care about active users.

Here's another build that does just that as well as adds proper handling of ProcessCanceledException. Please let me know if what difference it makes.

I'm not going to include this in tomorrow's build but would love it if you could continue to help validate this since I don't have access to an org with nearly this level of user data.

[Kris Goncalves]

Upon clicking on the dropdown, I get an error saying

“org.apache.cxf.binding.soap.SoapFault: INVALID_FIELD:
Name, Username FROM User WHERE IsActive = TRUE
^
ERROR at Row:1:Column:43
No such column 'IsActive' on entity 'User'. If you are attempting to use a custom field, be sure to append the '__c' after the custom field name. Please reference your WSDL or the describe call for the appropriate names.”

Not quite sure why as the field is “IsActive”.

‌

[Scott Wells]

It's likely because I'm querying it using the tooling API instead of the partner API and that field isn't available in the tooling API. Fun, fun. Hold tight and I'll see if I can cut it over to the other API.

[Scott Wells]

Take 3. The User query now occurs via the partner API, and I've verified that it properly executes vs. before where I just added that clause and posted a new build blindly assuming all would be well. Sorry for the false start, and thanks again for helping to test these changes!

[Kris Goncalves]

Took around 30 seconds this time around. Much better, thanks! Confirmed cancel button works as well without any errors.

[Scott Wells]

Hey, that's an improvement! Obviously not ideal, but based on the information you've provided, it sounds like that's kind of on the extreme side in terms of row counts. I'll button that up nicely for tomorrow's build as it's not really much of a change (logic-wise, I mean). I think the next optimization would be to show non-community users by default with an option to show community users since that's likely more the exception. Let's see get this first step out there first, though... Thanks again!

[Scott Wells]

Delivered in 2.1.7.6. If this still ends up being insufficient, let's log another issue for further optimizations.