Deployment fails due to SF lock, possibly caused by validation, then IC thinks file is up-to-date

[June Bischoff]

I'm pretty regularly running into issues trying to deploy my changes to Salesforce and having the deployment fail because of the SF admin setup lock.

11:58:45,072 [78646199]   INFO - er.ForceComMetadataApiDeployer - Starting deployment with a polling interval of 5000 milliseconds. 
2016-06-16 11:58:50,483 [78651610]   INFO - er.ForceComMetadataApiDeployer - Deployment status = FAILED 
2016-06-16 11:58:50,483 [78651610]   WARN - .IlluminatedCloudExceptionUtil - Deployment failed (ALREADY_IN_PROCESS) - null: The changes you requested require salesforce.com to temporarily lock your organization's administration setup. However, the administration setup has already been locked by another change. Please wait for the previous action to finish, then try again later. 
2016-06-16 12:00:32,205 [78753332]   INFO - er.ForceComMetadataApiDeployer - Starting deployment with a polling interval of 5000 milliseconds. 
2016-06-16 12:00:38,300 [78759427]   WARN - der.ForceComToolingApiDeployer - No deployment result found.

This is partially due to an increase in the number of developers working in the same sandbox simultaneously. Whatever you can do to reduce the number and duration of the locks taken out would be helpful.

However, I'm also seeing this when I'm the only one working in the system. My Validation & Deployment settings are Compile with the Tooling API on Edit and on Save, and it seems like this is most likely (but not exclusively) to occur when a validation is running simultaneously with a deployment. If this is indeed the issue, can you make it so they don't collide?

Finally, when my deployment fails due to Salesforce lock, it seems to reset the timestamp on the file I'm trying to deploy, so it's quite likely to tell me the files are up-to-date on the second attempt. (They are not.) Then I get into a fun little cycle of making a minor whitespace change, waiting for the validation to complete, and hoping my deployment succeeds.

I appreciate anything you can do to make deployment run smoother! Let me know if I can provide additional information.

[June Bischoff]

Looking at the logs, I see another set of errors when deploying and failing for no apparent reason:

    at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

2016-06-16 12:20:12,024 [79933151]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 
2016-06-16 12:20:14,998 [79936125]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 
2016-06-16 12:20:15,048 [79936175]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 
2016-06-16 12:20:15,088 [79936215]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 
2016-06-16 12:20:15,138 [79936265]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 
2016-06-16 12:20:34,255 [79955382]   INFO - er.ForceComMetadataApiDeployer - Starting deployment with a polling interval of 5000 milliseconds. 
2016-06-16 12:20:39,421 [79960548]   INFO - er.ForceComMetadataApiDeployer - Deployment status = FAILED 
2016-06-16 12:20:39,431 [79960558]   WARN - .IlluminatedCloudExceptionUtil - Deployment failed (ALREADY_IN_PROCESS) - null: The changes you requested require salesforce.com to temporarily lock your organization's administration setup. However, the administration setup has already been locked by another change. Please wait for the previous action to finish, then try again later. 
2016-06-16 12:20:41,381 [79962508]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 
2016-06-16 12:20:41,461 [79962588]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 
2016-06-16 12:20:41,521 [79962648]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 
2016-06-16 12:20:50,053 [79971180]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 
2016-06-16 12:20:50,123 [79971250]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 
2016-06-16 12:20:50,173 [79971300]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 
2016-06-16 12:20:50,223 [79971350]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 
2016-06-16 12:21:15,626 [79996753]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 
2016-06-16 12:21:15,676 [79996803]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 
2016-06-16 12:21:15,736 [79996863]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 
2016-06-16 12:21:15,816 [79996943]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 
2016-06-16 12:21:29,867 [80010994]   INFO - er.ForceComMetadataApiDeployer - Starting deployment with a polling interval of 5000 milliseconds. 
2016-06-16 12:21:35,292 [80016419]   INFO - er.ForceComMetadataApiDeployer - Deployment status = FAILED 
2016-06-16 12:21:35,292 [80016419]   WARN - .IlluminatedCloudExceptionUtil - Deployment failed (ALREADY_IN_PROCESS) - null: The changes you requested require salesforce.com to temporarily lock your organization's administration setup. However, the administration setup has already been locked by another change. Please wait for the previous action to finish, then try again later. 
2016-06-16 12:21:43,672 [80024799]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 
2016-06-16 12:21:43,722 [80024849]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 
2016-06-16 12:21:43,771 [80024898]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 
2016-06-16 12:21:43,821 [80024948]   INFO - agnostic.FrequentEventDetector - Too many events posted, #1 

[Scott Wells]

Let me take a look, June. I thought I was putting all deployment operations - whether validation or actual deployment and independent of the API used - in a single execution queue. As a result the only reason I'd expect the type of lock failure you describe for a single user would be if the lock is held after the operation is reported as completed by Salesforce. It's possible there are places where that's not the case, though, so let me dig deeper.

As for this when multiple users are working in an org, I may be limited in what I can do, but I'll investigate a bit and see what options are available.

Finally, as for the file being marked incorrectly as no longer needing deployment, that's a bug for sure and I'll take care of it.

[Scott Wells]

I moved all of these operations into a single execution queue. Have you seen these types of problems anymore?

(obviously I'm doing a pretty serious scrub of old bugs and I imagine I'll resolve some too aggressively so let me know if this one...or any of the others reported by you...are still a problem)