Can't create a project – retrieval ALWAYS fails 10k-limit error

[Petr Švestka]

Hi,

I can't retrieve items when creating a project.

• With default contents settings ("All/Package"), it'll fail with limit 10000 exceeded.
• If I select only classes, it fails with the same error.

I tried to use a simple package.xml to retrieve all classes. But the plugin always rewrites it with an empty file and doesn't retrieve any.

Overview * We do have 10k+ reports, but they were never retrieved in the "All/Package" settings until a recent version. * Some 300 classes only.

As if it always downloads the whole project and fails when it realizes there's too many items.

Nothing useful found in the logs either. Idea version is 14.1.4 Community.

Please advise if there's a working way how to retrieve: objects, fields, classes, triggers, VF pages and components. This should be far less items than 10k.

Thank you.

[Scott Wells]

Hi, Petr. Sorry for the issues. You're correct that I recently fixed a bug that was keeping foldered metadata types including reports from being included in retrievals. Just so I understand, if you change the metadata selection from All/Package to Selected and choose only ApexClass, you see the same error message even though you only have ~300 classes?

[Petr Švestka]

Yes :-)

[Scott Wells]

Do you mind providing two things? First, a screenshot of the retrieval dialog just so I know the particular selection. Second, an idea.log with some additional debug info. You can get it to output additional logging by going into Help>Configure Debug Log Settings and adding the following lines:

#com.illuminatedcloud.intellij.builder.ForceComBuilder
#com.illuminatedcloud.intellij.builder.ForceComMetadataRetriever
#com.illuminatedcloud.intellij.builder.RetrieveAction

Thanks!

[Petr Švestka]

Hi Scott,

Set the debugging and tried to retrieve data via Build | Illuminated Cloud | Retrieve Metadata... in the project. Pls see the screenshot (apart from classes nothing was selected). It again ended with the 10k-limit error.

[Petr Švestka]

Idea log with debug statements on

[Scott Wells]

Thanks, Petr. Do you mind sending me the other idea.log files that were part of that operation? This one starts in the middle of processing the reports. Also, could you take a screenshot of the retrieve dialog itself and attach it? I want to see what retrieval scope and other options you have selected when this occurs.

[Petr Švestka]

Logs from Jan 6.

[Petr Švestka]

Logs attached.

Which retrieve dialog you mean? I thought the big screenshot in my post half an hour ago displays everything:-)

[Scott Wells]

Sorry, yeah...there are a few to choose from! The one that says "Select Retrieval Scope" in response to a request to retrieve. I want to see what scope you've selected (Project vs. Module + Dependencies vs. Module vs. Selected vs. Custom), whether you've selected Include Missing Metadata, etc.

[Petr Švestka]

Attached as instructed

[Scott Wells]

Okay, that helps. I think I see why this may be happening. Give me just a few and I might have a workaround for you until I can provide a real solution. More to come shortly!

[Scott Wells]

Yeah, I have a workaround to get you unstuck, but as you'll see from the workaround, I definitely have something to fix here as well! Basically just create a dummy class from within IC. Right-click on your src node in the Project tree and click New>Apex Class. I called mine DeleteMeAsap to be clear that it needs to be removed as soon as I'm past the issue. After that dummy class deploys, you can retrieve classes. Once classes have been retrieved, you can delete the dummy class. You might need to repeat this for other metadata types that you need to retrieve. Hopefully this gets you unstuck. I'm VERY sorry for the issue. I'll try to get this fixed for real ASAP as the workaround is frankly embarrassing!

[Scott Wells]

Petr, I found the issue...literally a closing brace placement in these most recent changes! I can deliver a fix over lunchtime to make this work the way you'd expect. I'll resolve this issue once I've delivered that.

[Petr Švestka]

Cool! This helped, thank you from a prompt workaround.

After classes were retrieved, I was able to successfully retrieve objects, fields, pages, components and triggers – I think all that's needed for normal work. But, please, I'll be waiting for a permanent fix:-)

Also, do I get it right that from now on, if we have more than 10k items in an org I can't create a project for it with the default "All/Package" settings and must pick cautiously? Just so that I know.

[Petr Švestka]

Good stuff Scott! Once again many thanks for quickly resolving it.

[Scott Wells]

Yeah, right now it doesn't handle retrieval of >10K items. This is related to issue #163 which deals with a different variation on the same general problem. I'll look at taking care of that once I put these other fires out.

[Scott Wells]

Fix delivered in 1.6.7.9.

[Scott Wells]

Hah! Actually it looks like you opened the current ticket on the 10K item limit as issue #107.

[Petr Švestka]

Yeah, once I need those items and then no:-)

If automatic "batchization" is too complicated now, I can live with some workarounds once I know of them.

E.g., when I need all reports, then I fetch them semi-automatically, like a–k, l–p, and q–z. Or, when I know the org has > 10k items and I'm creating a project, I know I need to cautiously select only what I need and not "All" (which worked until recently).

:-)

[Scott Wells]

Unfortunately the previous behavior was based on a bug that was causing some other users not to be able to retrieve foldered metadata. Hopefully the latest update I just uploaded will put you back into a good situation where you just won't be able to use "All/Package" against orgs with >10K metadata items; instead you'll need to use "Selected" and choose everything except for, in your case, Reports, and then you can retrieve reports selectively.