OST generation fails with "Status code 500 returned trying to retrieve .../completions/?type=apex: Server Error" (and slower OST generation due to system classes query)

[Scott Wells]

I’m opening this as a way to communicate with users proactively about a set of issues in the Salesforce API used by IC2 (and other third-party tooling) to query the Salesforce standard/system Apex types.

That API call has historically been very quick, typically sub-second or a small number of seconds to complete. However, over time it has become slower and slower, sometimes taking over a full minute to complete. Additionally it has recently started to fail, though it will eventually succeed after one or more retries. Failure manifests as follows:

‌

I am in active discussion with Salesforce about these issues. They have identified the root causes (there are evidently multiple contributing problems) and are investigating potential solutions. I will update IC2 as appropriate based on these solutions (if updates are even necessary) and will post progress as part of the standard release notes.

UPDATE: It appears that as part of a temporary workaround for these issues, Salesforce has removed the Slack namespace from the completions?type=apex API response. If this is affecting you, please let me know so that I can provide that feedback to Salesforce.

[Wannes]

Had the same issue today

[Scott Wells]

Hi. Unfortunately it’s a known issue in that Salesforce API. Evidently Spring '24 includes some more advanced fixes than what have been delivered so far, both for the 500 and for the slow API speed.

[Scott Wells]

For those of you seeing this more frequently all of a sudden, specifically in sandbox orgs, I’ve reported it to Salesforce who has provided the following:

Thanks for raising this up Scott. I tracked down the CS127 org 00D3M0000000axd logs. It is failing with a GACK - 1792059633-84888 (-22959106)There appears to be a missing class. I'll raise a bug internally for it.

Given the nature of the bug, I’m hoping that it will be patched with the approprivate sense of urgency. I will post updates here as they provide more information.

In the interim, if you’re experiencing this, I’ve attached a boilerplate OST archive here. Please see the comment associated with its attachment for instructions on how to download and use it.

[Scott Wells]

I'm attaching a boilerplate OST archive from a stock standard Winter '24 scratch org that folks encountering this issue can download and use until Salesforce has issued a patch.

Download the archive (do not extract it) and place it in the affected project(s) under IlluminatedCloud/<connectionName>, then close (if necessary) and reopen the project. IC2 should begin using the OST archive.

Note that if your orgs have extended features enabled, this OST may not contain the standard Apex classes for those features. Your best option in that case would be to create a new scratch org with those features enabled and generate an OST from that scratch org, then drop it in place of the one provided above.

I will post here when the issue has been resolved by Salesforce at which point you should be able to generate a proper OST from these projects' associated orgs.

[Scott Wells]

For those of you having this issue, let's try a different workaround. I'm attaching a build that, upon receiving an HTTP 500 in response to the Tooling REST completions?type=apex API, uses a "canned" response instead and proceeds with OST generation.

If you're affected by this issue, can you please install this build by downloading the attached IlluminatedCloud2.zip archive (again, do not extract it) and installing it using Settings | Plugins | Install plugin from disk (under the gear drop-down menu)? Then please try to generate an OST in one of the projects affected by this issue and see whether it completes.

Note that the bundled API response is for a quite vanilla scratch org, so feature-specific standard Apex types won't be included. I'm currently trying to create a "kitchen sink" scratch org with as many features enabled as possible so that I can get a response for this API from that org that will hopefully include more of a superset of what people actually need/use than a potential subset.

Anyway, please let me know whether this helps or not. If so, I'm planning to include it in the next weekly build to provide a reasonable workaround until Salesforce fixes the underlying API issue.

[Scott Wells]

Delivered in 2.2.9.7.