MALFORMED_QUERY: SOQL statements can not be longer than 20000 character

[harmon]

When I click the "Illuminated Cloud > Retrieve Metadata..." context menu option, I am getting this error, and I'm not able to move past it. I am using the package.xml to specify which metadata I want to retrieve. I tried deleting a few entries in the package.xml in case it was using that to do queries against the SF API, but that did not work.

Any ideas on how I can get past this? Is it related to our Org having lots of objects/fields?

Thanks, Adam

[Scott Wells]

Is there a detailed stack trace in the log that you might be able to send? That would let me know exactly which SOQL query is being generated in excess of 20K characters.

[harmon]

Hi Scott,

As a developer, I apologize for not attaching any! I went back and read how to do that.

2016-11-11 14:06:20,692 [  43283]   INFO - tor.impl.FileEditorManagerImpl - Project opening took 22486 ms
2016-11-11 14:08:09,535 [ 152126]   INFO - y.ReflectionServiceFactoryBean - Creating Service {urn:partner.soap.sforce.com}PartnerService from class com.sforce.soap.partner.PartnerApi
2016-11-11 14:08:17,267 [ 159858]   INFO - y.ReflectionServiceFactoryBean - Creating Service {urn:partner.soap.sforce.com}PartnerService from class com.sforce.soap.partner.PartnerApi
2016-11-11 14:08:17,822 [ 160413]   INFO - y.ReflectionServiceFactoryBean - Creating Service {http://soap.sforce.com/2006/04/metadata}MetadataService from class com.sforce.soap.metadata.MetadataApi
2016-11-11 14:08:45,368 [ 187959]   INFO - y.ReflectionServiceFactoryBean - Creating Service {urn:tooling.soap.sforce.com}ToolingService from class com.sforce.soap.tooling.ToolingApi
2016-11-11 14:09:05,646 [ 208237]   WARN - .IlluminatedCloudExceptionUtil - MALFORMED_QUERY: SOQL statements can not be longer than 20000 characters.
javax.xml.ws.soap.SOAPFaultException: MALFORMED_QUERY: SOQL statements can not be longer than 20000 characters.
    at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:160)
    at com.sun.proxy.$Proxy162.query(Unknown Source)
    at com.illuminatedcloud.client.ToolingQueryResultProcessor$1.compute(SourceFile:36)
    at com.illuminatedcloud.client.ToolingQueryResultProcessor$1.compute(SourceFile:32)
    at com.illuminatedcloud.client.ForceComApiClient.runWithClient(SourceFile:181)
    at com.illuminatedcloud.client.ToolingQueryResultProcessor.process(SourceFile:31)
    at com.illuminatedcloud.intellij.builder.ForceComBuilderUtil.getFileProperties(SourceFile:420)
    at com.illuminatedcloud.intellij.builder.ForceComBuilder.createAllMetadataPackage(SourceFile:2737)
    at com.illuminatedcloud.intellij.builder.BuildOptionsDialog.getMetadataContents(SourceFile:1607)
    at com.illuminatedcloud.intellij.builder.BuildOptionsDialog.access$1000(SourceFile:111)
    at com.illuminatedcloud.intellij.builder.BuildOptionsDialog$9.run(SourceFile:628)
    at com.intellij.openapi.progress.impl.CoreProgressManager$TaskRunnable.run(CoreProgressManager.java:635)
    at com.intellij.openapi.progress.impl.CoreProgressManager$9.run(CoreProgressManager.java:384)
    at com.intellij.openapi.progress.impl.CoreProgressManager$3.run(CoreProgressManager.java:170)
    at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:494)
    at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:443)
    at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:54)
    at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:155)
    at com.intellij.openapi.application.impl.ApplicationImpl.lambda$null$9(ApplicationImpl.java:548)
    at com.intellij.openapi.application.impl.ApplicationImpl$2.run(ApplicationImpl.java:307)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.cxf.binding.soap.SoapFault: MALFORMED_QUERY: SOQL statements can not be longer than 20000 characters.
    at org.apache.cxf.binding.soap.interceptor.Soap11FaultInInterceptor.unmarshalFault(Soap11FaultInInterceptor.java:86)
    at org.apache.cxf.binding.soap.interceptor.Soap11FaultInInterceptor.handleMessage(Soap11FaultInInterceptor.java:52)
    at org.apache.cxf.binding.soap.interceptor.Soap11FaultInInterceptor.handleMessage(Soap11FaultInInterceptor.java:41)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307)
    at org.apache.cxf.interceptor.AbstractFaultChainInitiatorObserver.onMessage(AbstractFaultChainInitiatorObserver.java:113)
    at org.apache.cxf.binding.soap.interceptor.CheckFaultInterceptor.handleMessage(CheckFaultInterceptor.java:69)
    at org.apache.cxf.binding.soap.interceptor.CheckFaultInterceptor.handleMessage(CheckFaultInterceptor.java:34)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307)
    at org.apache.cxf.endpoint.ClientImpl.onMessage(ClientImpl.java:802)
    at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponseInternal(HTTPConduit.java:1673)
    at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponse(HTTPConduit.java:1551)
    at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1348)
    at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)
    at org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:651)
    at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307)
    at org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:516)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:425)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:326)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:279)
    at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:96)
    at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:138)
    ... 24 more

[Scott Wells]

That helps! Do you happen to have a large number of process builder flows in your org? I've had to implement a workaround for an issue with metadata retrieval around flows, and it looks like the query for that workaround is what's blowing up here. I can definitely change that to batch in a way that each query will fit into 20K characters. Just want to make sure that what I'm seeing matches what you know to be in your org.

[harmon]

Yeah, I'd say we probably have a bunch of process builders/flows. We have like 4 pages of process builders...

[Scott Wells]

Okay. Thanks for confirming. I'll add batching to that query for a build coming out very early next week. For the moment you can work around this for other metadata types by removing flows from your subscription.

[Scott Wells]

Can you try out this test build that batches through loading of flows to see whether it resolves the issue? Just download the archive and install it using Settings>Plugins>Install plugin from disk. Please let me know...

[harmon]

Yeah, that worked! (The zip you sent me) Thanks man, I appreciate the fast response. I did not have any references to Flows in my package.xml, btw:

        <name>ApexClass</name>
        <name>ApexComponent</name>
        <name>ApexPage</name>
        <name>ApexTrigger</name>
        <name>BusinessProcess</name>
        <name>CustomField</name>
        <name>CustomObject</name>
        <name>CustomPermission</name>
        <name>GlobalValueSet</name>
        <name>ListView</name>
        <name>RecordType</name>
        <name>StaticResource</name>
        <name>WebLink</name>

[Scott Wells]

Yeah, it's happening to figure out what metadata is available before reconciling with your own subscription. I might be able to make it smart enough to avoid that if it's expensive, but my guess is that's not really where the time is going. Glad it fixed it, though! I'll have this in the formal release on Monday, so for now just hang onto that test build.

[harmon]

Cool, thanks! You're IntelliJ plugin has REALLY cut down on the time I spend waiting on compiling/running tests! Really awesome product, thanks!

[Scott Wells]

Delivered in 1.7.5.5.