Anonymous SOQL Completion not working correctly

[Jason Clark]

When using the anonymous SOQL editor, completion isn't working as expected. For example, I entered a simple query outline:

select id
from Transaction__c
limit 1

I ran the query, it works. Then I added a comma after "id", expecting to see a list of fields on the Transaction__c object (based on screen shots found here). No completion window appeared. I tried Ctrl+Space, and no completion window appeared.

At this point I started googling in another window; after 15-20 seconds had passed I noticed the the completion window had popped up. However, the completion list appears to contain all possible sObject fields, not just those of the object being queried:

I tried a full rebuild of offline symbol tables an a restart of IDEA; this didn't impact the observed behavior.

[Scott Wells]

Jason, do you see any errors/exceptions in your idea.log correlated with this behavior? I just tried to reproduce it and wasn't able to, so hopefully there will be something telling in the log.

[Jason Clark]

I see lots of "WARN - ntellij.ui.tree.AsyncTreeModel - ignore duplicated child at...", but nothing that jumps out at me. I also don't have any debug logging enabled; let me know if there's a feature (or features) I should enable. I tried enabling completion (ctrl+space) while watching the log live (via console.app), no new entries were created. I will attach my current log to the ticket in case it's useful.

[Jason Clark]

Current idea.log as requested. Let me know if you'd like a clean one with any particular steps carried out or debugging options enabled.

[Scott Wells]

Thanks, Jason. I'm not seeing anything telling in that log. Those warnings are curious but are coming from plugin SDK code.

Do you mind rebuilding your caches & indices to see if that helps with either/both of these? You can either do that for just IC's indices under Illuminated Cloud>Configure Project, or you can do it for everything in the IDE using File>Invalidate Caches / Restart.

[Jason Clark]

I tried the Invalidate & Restart option; no change. I then tried another Generate Symbol Tables & restart, no change. I've attached the updated log in case it's helpful.

This is a new IntelliJ install & Illuminated Cloud install as of 4 days ago. I did start with an existing project (was using Mavens previously) and followed the prompts to "fix" it (add org connection, etc). I'm brand new to this IDE so perhaps I missed something?

[Scott Wells]

Okay. Thanks for letting me know. Let me see if I can figure out what might cause this repeated "ignore duplicated child at ..." warning to be emitted. Given that I've never seen it before, I have to assume it's related to that somehow.

As for how you created the project, what you describe sounds fine. You should be able to point it at an existing directory structure, version control repository, or org and have it build a project around it for you. I'm assuming you did that using File>New>Project from existing sources?

[Jason Clark]

Thanks Scott. Actually, I don't think I saw Project from existing sources; On first launch IntelliJ gives you a splash page, I believe i tried "Open..." and then just selected my existing project directory. Once indexing was finished, I got a pop-up for the event log message, "Invalid configuration for module 'xxx'". I used the resolve link and followed the instructions... think I may have had to restart once or twice, but eventually I stopped getting event log errors and things started working as expected.

I have noticed a couple of other odd behaviors, I only mention them here in case they are symptoms of this issue. If unrelated I can open other tickets later.

1. In some (but not all) Apex Class files - mostly the larger files - All of the entries in the Stucture (outliner) view are doubled. Every method, property, constant appears twice.
2. When opening files with Navigate > File... (Cmd-Shift-O), using the Filter button to remove meta.xml files doesn't work.

I'm willing to restart this project using File>New>Project from existing sources if you think it will help; but should I do any cleanup beforehand? E.g., remove .idea/ and IlluminatedCloud/ directories from the project directory?

[Scott Wells]

Jason, Open should have provided the same result as File>New>Project from existing sources, so recreating the project that way shouldn't make a difference. Having said that, where is this metadata mastered? Is it managed under version control? in the org? I ask because we might want to recreate the project from its master location just to see if we get different results, allowing IC to lay things itself.

Creating a project from existing sources shouldn't present an issue assuming the project follows general standard conventions for Salesforce projects, but given the strange behavior you're seeing, it may be worth ruling out something strange being picked up from the current project.

As for the other two things you're seeing, the duplicate items in the outline view has been reported and I'm planning to provide a fix in the next build. I had never seen the issue with filtering of meta.xml files in the open files dialog, but I did just reproduce that. Might be worth logging a distinct bug for that if it's an issue and I'll see what I can do.

[Jason Clark]

The project is managed via Git repo, and consists of all Classes, Components, Labels, Pages, Static Resources, Triggers, and AuraDefBundles in the org (with package.xml to match).

[Scott Wells]

Okay. Please recreate it using File>New>Project from Version Control in a separate folder and see if that leads to the same or different (ideally correct!) behavior. If it's the same, it must be something about the way things are laid out in the project itself. If it's different/fixed, we can diff the two and see what caused the original issues.

[Jason Clark]

Done. File>New>Project from Version Control; got the same warnings (forgot to take notes, but I think it was add a connection, configure module, generate symbol tables). No change:

The first time I tried to generate symbol tables I did get an error:

but then it restarted IDEA, I got the warning to generate symbol tables again, and it worked fine that time.

[Scott Wells]

Okay. What happens if you use the SEL or SEL* live templates with the Transaction__c object? If you're not familiar with those, just type SEL and then hit Tab, then in the drop-down of SObject type names, choose Transaction__c. It will start with the Id field, and then let's see if you can add more fields after it. Also try SEL* which works the same but does an effective SELECT * on the object.

I'm also wondering if perhaps that object name itself might be coming into play here since transaction is a reserved word, but let's rule things out one at a time.

[Jason Clark]

Interesting aside: Typing SEL shows the possible completions, including SEL and SEL* live templates, but as soon as I type *, the completion window disappears. Ctrl+Space brings it back, at which point I can press tab or enter to choose the SEL* live template.

So: SEL * live template works for both Transaction__c as well as Booked_Loan__c. All fields retrieved, looks great.

SEL live template shows the sObject list correctly (editing FROM clause), but once selected, the field completion is still broken, just as before. Tested with Booked_Loan__c as well as Transaction__c. Screenshot below is SEL<tab><choose Booked_loan__c><tab>, leaving id field selected:

[Scott Wells]

Okay, something VERY strange is going on. Let's get more fundamental. What version of IntelliJ IDEA/WebStorm are you running?

[Jason Clark]

AFAIK, latest available - downloaded and installed IDEA & IC last Saturday.

[Scott Wells]

I figured as much but wanted to rule out any potentially low-hanging fruit. Okay, next step on our long journey to debug this is to gather diagnostic info from code completion itself. Please enable debug logging for Apex Completion as documented here:

http://www.illuminatedcloud.com/support/debuglogging

reproduce the issue, and send me the resulting idea.log so I can see which completion rules are and aren't firing and why. If that doesn't shed any light we may need to have a quick screen sharing session so that I can take it a bit more off-road. This one is definitely a stumper so far!

[Jason Clark]

Well at least I'm not boring! Added Apex Completion debug logging per your link, tried SEL live template; new log attached.

[Scott Wells]

Thanks for the quick turnaround, Jason! Let me review the log and I'll let you know what I find (fingers crossed).

[Scott Wells]

Okay, so it does look like it should be offering completions for the fields of that SObject:

2018-04-20 10:15:33,463 [71554785]  DEBUG - mpletion.ApexCompletionContext - Looking for completions where contextElement = SOQL_QUERY, currentElement = IDENT, beforeElements = [SELECT], afterElements = [SOQL_FROM_CLAUSE, FROM, SOQL_FROM_OBJECT, SOBJECT_TYPE, IDENT] 
2018-04-20 10:15:33,464 [71554786]  DEBUG - .completion.ApexCompletionRule -   Soql.Select rejected because previous element must be one of [null, [, (] 
2018-04-20 10:15:33,476 [71554798]  DEBUG - mpletion.ApexCompletionContext -     Rule Soql.Fields added 167 completions 

I'm going to ask a dumb question here that I probably should have asked from the get-go...is there any chance that you're expecting completions to work in a case-insensitive manner by default? For example, are you trying to complete the Name field by typing name and it's not showing up? If so, does it show up when you type 'N' to start the field name instead of 'n'? If that's the case, I can show you how to make code completion case-insensitive in the IDE which may resolve the issue. If that's not the case, I do think the next step would be a screen share if you're okay with that.

[Jason Clark]

Just ran into something weird. Maybe unrelated, but feels like it could be an indication that something isn't recognizing the project as "salesforce", so thought I'd share.

Was going to try "Code Cleanup", and test it on one file. Analyze > Code Cleanup..., set a custom scope for "Current File" (which is an Apex class file), and then I clicked the ellipsis button next to the Inspection Profile picklist; this is the result:

This is the 'project default' profile but I checked IDE default, it's the same. I thought it was odd that the Salesforce section was missing. To be sure, I checked the project defaults in settings (Settings > Editor > Inspections) and Salesforce is certainly in there:

Didn't see an existing issue, so don't know if this is a symptom of my issue or unrelated, but felt likely enough to add here.

[Scott Wells]

Interesting...unfortunately I see the same thing when I follow those exact steps. The reason is that IC doesn't include any inspections that are designated as code cleanup inspections, though a few almost certainly should be. If you use Analyze>Inspect Code or Analyze>Run Inspection by Name the IC inspections will be included.

It does seem that your project is configured as an IC project or almost all of the other features would be disabled. However, something strange is definitely going on here. No doubt about that!

[Jason Clark]

OK, Hang on. Apparently I am blind, because I just retested (based on your last question on case-sensitive completion) and realized I gave the wrong answer yesterday. I've retested 3 times to make sure I'm not crazy. I also looked at the screenshot from this comment yesterday, and realized that I completely got it wrong yesterday.

So. SEL<tab><Choose Transaction__c><tab> produces the bad completion. Note how each item in the list has its file name in parens:

But. SEL<tab><Choose Booked_Loan__c><tab> produce the RIGHT completion. Note how each item does NOT have a file name in parens:

I don't know how I missed this yesterday; yesterday's screenshot linked above shows the same thing. Tested with other custom and standard objects as well. At this point, I think we can say your prior guess was correct... Transaction__c must be the culprit, perhaps because of the name being so close to a reserved word. Very sorry for dragging this out longer than needed.

But! There is a case-sensitive thing. Here's SEL<TAB><Choose Booked_Loan__c><Tab><Delete default "Id"><Type lowercase "i">:

Do you want another log with Apex Completion debugging turned on, using Transaction__c as the example? Pretty sure I used Booked_Loan__c for the last one.

Finally - would love to try case-insensitive completions, since I routinely ignore capitalization while typing... can you point me at that? Thanks for all your help & patience on this.

[Scott Wells]

Hey, we're getting some traction now! Thanks for verifying all of that. So, you can configure the IDE for case-insensitive code completion under Settings>Editor>General>Code Completion by changing the value for Case-sensitive completion to None. Note that that will affect ALL languages in the IDE, though. I have an open issue right now for making only IC's languages case-insensitive for code completion, but until I take care of that, you can use that setting to do it.

As for Transaction__c being the culprit, it definitely seems like it could contribute, but coincidentally the main project in which I'm working right now also happens to have a Tranasction__c object and I'm not seeing the same issue. However, it's worth pointing out that I'm working in a namespace to the full name is <namespace>__Transaction__c, so it's not quite apples-to-apples.

Let me create a test Transaction__c object in an org without a namespace and see if I can finally reproduce what you're seeing. Hopefully I'll be able to and then it's just a matter of debugging and fixing the underlying issue. At least now I think we've managed to narrow it down significantly!

[Scott Wells]

Oh, and yes, if you want to do the same thing with Transaction__c and provide that log, it might help to accelerate things. Thanks for offering!

[Jason Clark]

Here's an updated log with Apex Completion debugging enabled, trying to complete a query for Transaction__c

[Jason Clark]

BTW, setting Case-sensitive complete to None fixes the SOQL completion issue shown here for Booked_Loan__c:

That looks like exactly what I'm looking for.

[Scott Wells]

Glad to hear! And based on the latest provided log, it does look like Transaction__c is causing some weird behavior:

2018-04-20 11:26:17,753 [75799075]  DEBUG - mpletion.ApexCompletionContext - Looking for completions where contextElement = SOQL_QUERY, currentElement = IDENT, beforeElements = [SELECT], afterElements = [SOQL_FROM_CLAUSE, FROM, SOQL_FROM_OBJECT, SOBJECT_TYPE, IDENT] 
2018-04-20 11:26:17,753 [75799075]  DEBUG - .completion.ApexCompletionRule -   Soql.Select rejected because previous element must be one of [null, [, (] 
2018-04-20 11:26:19,403 [75800725]  DEBUG - mpletion.ApexCompletionContext -     Rule Soql.Fields added 35760 completions 

Notice that it thinks that it needs to add almost 36K fields! I'll see if I can reproduce that, but I think we've managed to corner it now.

I also noticed that you're bumping up against the IDE's memory limits. The default max heap for IntelliJ IDEA is kind of low for any real dev (Salesforce, Java, or whatever). I'd recommend that you raise the max heap to 2GB or more to avoid bouncing against a memory ceiling when heavy things are happening. Here's a doc on how to do that:

https://intellij-support.jetbrains.com/hc/en-us/articles/206544869-Configuring-JVM-options-and-platform-properties

I'll keep you posted on my attempts to reproduce this now. Thanks again for the sleuthing help!

[Scott Wells]

Well darn it, I tried to reproduce this in isolation...no namespace, etc., and it didn't reproduce for me. Here's what I got trying the exact same thing:

2018-04-22 11:18:17,865 [179603540]  DEBUG - mpletion.ApexCompletionContext - Looking for completions where contextElement = SOQL_QUERY, currentElement = IDENT, beforeElements = [SELECT, SOQL_SELECT_FIELD, SOBJECT_FIELD_NAME, ID, ,], afterElements = [SOQL_FROM_CLAUSE, FROM, SOQL_FROM_OBJECT, SOBJECT_TYPE, IDENT] 
2018-04-22 11:18:17,865 [179603540]  DEBUG - .completion.ApexCompletionRule -   Soql.Select rejected because previous element must be one of [null, [, (] 
2018-04-22 11:18:17,867 [179603542]  DEBUG - mpletion.ApexCompletionContext -     Rule Soql.Fields added 31 completions 

And those 31 completions are in fact the fields of the Transaction__c SObject. Let me add some diagnostic logging to the code that lists fields on the SObject since that's now decisively cornered as the place where things are going awry for you. I'll include that in the next build (or perhaps even a test build here) so that we can continue our search for the ultimate root cause of this in your project.

More to come soon...

[Scott Wells]

Okay, here's a build with a bit more debug logging around how it determines the SObject type for which field completions should be offered. Install it using Settings>Plugins>Install plugin from disk. Once installed add the following additional two lines to Help>Debug Log Settings and reproduce the issue:

#com.illuminatedcloud.intellij.completion.ApexSObjectFieldsCompletionRule
#com.illuminatedcloud.intellij.psi.util.SObjectUtil

then please send me the resulting idea.log. That should provide a bit more info on why this is happening.

[Jason Clark]

Sorry, been swamped on a project, and as of yesterday, had to stop using IntelliJ, as it's consistently using 100-200% CPU and leaking memory. Flush caches & restart didn't fix it. Will try to figure it out this weekend and then I'll try the new debug build. Sorry for the delay!

[Scott Wells]

No problem. Man, you weren't kidding when you said you're not boring! Kidding, of course...I'm so sorry for the continued issues. As before, I definitely recommend making sure that you're on the absolute latest build of the base IDE, now 2018.1.2. I've debugged a few issues with folks recently including a major performance issue that were resolved by updating to the latest build of IntelliJ IDEA or WebStorm as appropriate.

If you continue to see these issues, let's just schedule a screenshare and see if we can get closer to the root cause and/or a solution.

[Jason Clark]

Thanks for the tip- I think I saw the update notice and dismissed it at the time as a task for later.. Just updated and so far everything seems to be back to normal. And thank goodness- can't believe how quickly I've gotten used to the time savings from IC!

So while updating I also bumped IC version to 2.0.1.7... does that build have the new debug log options, or do I still need to install the attached build? If I do install, will it automatically supersede the currently installed version, or do I need to disable that first? Will probably be tomorrow before I can test it.

[Scott Wells]

Okay, whew! I'm actually about to release 2.0.1.8 which (I'm pretty sure) has the extra debug logging. I would stay on the latest official version at this point, and if we find that we're missing the additional logging I'll just do a quick rev to add it back.

[Jason Clark]

Here's a couple of new log files generated with IDEA 2018.1.2 and IC 2.0.1.8 (auto-installed version), and with these debug options:

#com.illuminatedcloud.intellij.completion.ApexCompletionContext
#com.illuminatedcloud.intellij.completion.ApexCompletionRule
#com.illuminatedcloud.intellij.completion.ApexVariantsCompletionRule
#com.illuminatedcloud.intellij.completion.ApexSObjectFieldsCompletionRule
#com.illuminatedcloud.intellij.psi.util.SObjectUtil

I tested with SEL<TAB>Transaction__c<TAB> and still see the bug. This should all be in Idea.log.1.

However - the CPU/Memory issue I referenced yesterday returned a couple hours after upgrading to IDEA 2018.1.2. After a little trial and error, I found that closing my project and opening another would stop the issue, but going back to my project would return the issue; I further found that closing the Structure window and restarting resolved the issue. While I normally leave it open always, I worked all day yesterday without it and had no further issues.

I opened Structure today and tried opening a couple of project files. No return of error, and memory use was under 1 GB. After a while, I opened a third file, and CPU jumped and stayed between 100%-200%. Also, memory use started climbing immediately. I closed IDEA at 1.78GB; although I've capped the VM at 2 GB, I saw it climb over 2.7 yesterday during testing. I closed the Structure pane, no change, so I restarted IDEA (which led to a new log being generated). With the Structure pane closed, the same three files cause no issues, nor do additional files. Re-opening Structure (with 4 files open) didn't cause the issue to re-occur, nor did opening another file. It's got to be some combination of time passing and switching/adding Structure contexts.

Also noticed: those "ignore duplicated child at 1: XXX" messages you noticed last week appear when I switch active files in the editor, but only if the Structure tab is open. Also, when the issue occurs, no extra logging occurs, except that eventually you get a Low Memory warning.

Sorry if this is a ramble. No one else seems to be having this issue, so I'm guessing (hoping) it's related to whatever in my project is causing the weird completion issue, so I thought it may be worth mentioning here.

[John De Santiago]

I am having issues with SEL* [TAB] not working so I came here to see if anyone else had issues. I noticed that I can use SEL* [ENTER] and it works but [TAB] doesn’t work for me anymore. I noticed this issue a few releases ago on my end before upgrading to 2021.2.2. Once things look OK enough to upgrade I did and the issue is still present. I thought it might work itself as being an issue related to 2021.2.2 but now I am not so sure. I don’t get any errors or logs. Wasn’t a big issue for me just an annoyance with something I use for both the SOQL query tab and in Apex.

[Scott Wells]

Hi, John. Just pinged you directly via Slack, but I'll put the same response here. I'm not seeing that behavior locally, nor have I heard about it from anyone else, so my guess is that it's either a local config issue (hopefully), or it's a bug that occurs due to something contextual in your environment. The first place I'd look is Settings/Preferences > Editor > Live Templates > Apex and see whether the entry for SEL* has been modified locally. If so, it will be blue; if not, it will be black. If it has been modified, you can right-click and click Restore defaults. If it hasn't been modified, let me know and we'll see if we can corner whatever is going on.

[John De Santiago]

Yeah so looks like this was my issue. My global default for expanding templates was set to [Enter]. I must of accidentally changed that at some point. I changed it to [TAB] and I am a much happier person for it . Thanks for the quick response

[Scott Wells]

Doing a little (pre-)spring cleaning. Jason, are you still having this issue? If not I'll resolve this to clear it from the active list.

[Jason Clark]

@Scott Wells I just tried several scenarios, and I cannot repro. I’d say it’s good to close.

[Scott Wells]

Thanks for confirming. If you do see similar, either reopen this or log a new issue.