Sequentially run jobs in a branch

In some instances it may be that a user makes multiple commits to a branch in a short amount of time, causing multiple Workflows to execute at the same time, possibly deploying in an incorrect order. "Auto Cancel Redundant Builds" may not be suitable for all situations in which multiple deployments are expected. 

Solution: Enable a way to ensure that commits to a new branch are queued for sequential execution.

  • Guest
  • Jan 24 2019
  • Taking votes
  • Attach files
  • Jean-Paul Calderone commented
    March 29, 2019 16:46

    A related idea that this would seem to address is that two half-finished workflows are worth a lot less than one finished workflow.  Even without deploys, I would rather have CircleCI prioritize _finishing_ a workflow that it has started over _starting_ a new workflow.  Then, instead of hanging around waiting for complete results for two different pieces of work, I can wait for less time for one result and then act on it while CircleCI is producing the second result.

  • Admin
    Kunal Jain commented
    April 10, 2019 21:35

    Thank you so much for your interest in this feature. Initial straw-man that we have been discussing is to have serialization at a pipeline level or rather at workflow level. If pipeline/workflows are serialized, commits would build the order they were received by CircleCI. This feature is definitely on our radar but is currently backlogged behind some refactoring work. Happy to reach out once we start working on it. 

  • Adam Mills commented
    May 07, 2019 01:43

    This problem actively discourages people from increasing their concurrency (paying circleci), I'd think it would be a huge priority.

  • Guest commented
    July 10, 2019 13:14

    This is a big issue, since even the workaround like https://github.com/eddiewebb/circleci-queue are having problems. With that workaround there is still an edge case with workflows on job transitions not being seen (which is probably an API subtlety, since the API only lists jobs, not workflows)

     

    Neither can we build workaround with external locking, as there is no way of having a "on fail" trigger to clear the lock, so heavy timeouts need to be used - which then always block builds after failed ones.

     

    Any other known workarounds here? Or an update of this issue?

  • Steven Harman commented
    August 06, 2019 19:10

    We could really use this functionality to aide in our build/lint/test/deploy pipeline. Current work arounds to "Stacked Deploys" include Orbs like https://github.com/eddiewebb/circleci-queue, which suffer some race conditions. But they also mean a Workflow could be running, but effectively queued because it's in a `while true; sleep` loop. A built-in feature to the CircleCI platform to enforce serialization ought to be able to avoid the "running, but effectively queued" scenario.

  • Guest commented
    October 23, 2019 16:34

    This is an essential feature for us. The alternatives supported natively in CircleCI are not sufficient because they result in risky race conditions.

  • Thibaut LaBarre commented
    20 Feb 19:06

    Looking forward to this feature! We are also using https://github.com/eddiewebb/circleci-queue but that hogs a compute node while waiting. 

  • Daniel Gephardt commented
    24 Feb 16:01

    Builds within the same branch not running sequentially is a frequent problem for us as it leads to the "wrong" version of code being deployed to our testing environments. This feature would eliminate some back-and-forth between our QA organization and engineering