Dynamic config (aka: Setup Jobs)

Circle 2.0 has been great, but our team would like to be able to be able to control how the build pipeline operates at build-time by dynamically generating the circle.yml.

At build, we'd like to be able to somehow run a command that dumps out the content of a circle.yml. By doing so, we could calculate a build plan on our monorepo for the workflow that should be run specifically for what projects in the monorepo have changed on the branch. This would make it easier for us to run jobs in parallel and kick off a build flow that's much more dynamic.

  • Guest
  • Jan 29 2018
  • Taking votes
  • Attach files
  • j s commented
    02 Apr 23:34

    +1 I need this!

  • Guest commented
    14 Mar 22:09

    Any update?

  • Leo Yu commented
    14 Jan 18:55



    agreed with guest above that buildkite does this really well. The ability to generate configs at runtime would make our monorepo circleci configs SO much shorter.

  • Isabel Garcia commented
    October 16, 2019 10:10


  • Leo Yu commented
    August 13, 2019 20:45

    This is something that buildkite does really well - I hope you guys can implement something like this soon, or at least give us the ability to specify conditional branches of a pipeline when doing monorepo builds.

  • Nathan Dintenfass commented
    May 15, 2019 15:32

    @Ulrich Unfortunately, I don't have a date I can share. 

  • Ulrich Petri commented
    May 15, 2019 15:06

    Could you give an update for where this stands currently, or maybe even ETA?

  • Nathan Dintenfass commented
    May 15, 2019 13:44

    Pasting in details from a duplicate request (https://ideas.circleci.com/ideas/CCI-I-249):

    ```As a developer, I want the ability to generate my pipeline at runtime so that I can customize the stages and steps before execution. The idea being to "run this thing to generate the configuration (or config.yml) instead of hardcoding everything".

     A few reasons I think this is useful:

    1) Semi dynamic pipelines - not fully dynamic like Jenkins Pipelines, but more flexibility compared to now with what I would say is not much complexity for users

    2) Less templating - IMO templating in YAML files is confusing. Thanksfully, there isn't much now but I would be dissapointed.

    3) No need to invent additional YAML features. GitLab CI recently introduced the include directive, which I think is a massive mistake.

    4) One of my use cases is to build against multiple different platforms with fanout and fan in when those versions are not hardcoded into the YAML file.```

  • Sergei Mak commented
    February 26, 2019 07:35


  • Oron Riff commented
    February 25, 2019 16:51


  • Nathan Dintenfass commented
    April 10, 2018 19:35

    This kind of functionality is on our radar. Between now and then we are likely to offer other forms of dynamism that are closer to our current config paradigm (allowing for parameterized/shared bits of config as well as looking at possibly allowing filtering based on where files have changed to accommodate monorepos that want to run conditional workflows/jobs based on what parts of the repo are being changed). Along with that work will come a more formal schema for a fully-expanded form of our config that will be needed as a target for any dynamic config creation.