Build concurrency control for non-deployment steps

Issue #16262 open
Jim Zucker created an issue

Related to #12821 there is an need to queue builds so that publishing to Nexus of snapshots and artifacts happens in a predictable way.

Comments (11)

  1. Aneita Yang staff

    Hey @jim-ezops,

    Thanks for raising this - with the deployment concurrency control feature that we've released, we will automatically ensure that only one deployment is in-progress to each environment. If there is already an in-progress deployment to an environment, later deployments to the same environment will be paused. You can then manually resume the paused deployments, meaning that you can control the order in which your artifacts are built and deployed.

    Does this suit your use case?

    Aneita

  2. Jim Zucker reporter

    This use case is a bit different.
    As part of a Maven build you create artifacts and push them to an artifact repository with the 'deploy' target. In the case of java it pushes .jar files to the repository for use by other processes (this is not Continuous Deployment). In this use case you want all request queued and pushed normally.

    As this is a build process we would not do it normally in the deploy phase as it requires the full build env.

    In our case as we also have a repository behind the firewall, which is the case for most institutions/product companies, we have to build on AWS, so to orchestrate properly we would want to queue the requests.
    (We are currently researching how to build a queue in AWS due to this limitation)

  3. Brian Osborne

    +1 for the ability to queue builds. My use case is for deploying build artifacts to a development environment. The solution that was added - pausing future builds - works, but still requires a manual process of going to the pipeline builds and resuming paused deployments. This is not huge issue for us, just a nice to have.

  4. Matt Ryall

    @bkosborne - although this request mentions queuing, it is not solely about that, but also for concurrency control on non-deployment steps. The latter seems more significant, so I've renamed the ticket to focus just on that.

    Can you please raise a separate ticket for the queuing deployments bit so we can track it independently? Thanks!

  5. Aneita Yang staff
    • changed status to open

    Hey @jim-ezops,

    Thanks for the context and for explaining your use case. I can see why this feature would be useful to your and your team.

    We're currently working on higher priority requests so it is unlikely that we'll work on this anytime soon. In the meantime, I'll open this ticket so other users can express their interest in having this functionality.

    Aneita

  6. Shane Fast

    +1 for this. We have a custom master branch pipeline that occurs each time that a pull request is merged into our master branch. In this build, once all the tests pass and all the other checks are successful we tag the commit with a version number - this certifies to our team that this commit is good to proceed to the next stage in our deployment. This provides us with certainty that each and every change we make is stable, but as you might have guessed this means that we can only process one pull request at a time.

  7. Warren Mayocchi

    Any time external limited resources are invoked in a build, concurrency control is required ... publish artefacts, use a database, use sonar, and so on. Surprised pipelines does not already do this as this is a common requirement of a build server.

  8. Log in to comment