Allow env var interpolation

Users desire to use environment variable expansions in few places, namely `working_directory` and in the environment section.

Currently, we treat all values as literals and no interpolation is supported.

Frequent patters for desiring the feature is:

  • in `working_directory`, e.g. `/go/src/github.com/$ORGNAME/$REPONAME`
  • Related to paths, e.g. `PATH=/go/bin:$PATH`, `VERSION=0.0.$ {CIRCLE_BUILD_NUM}

`

  • Advanced uses, e.g. `TOKEN=$(heroku auth:token)`

For working_directory, the workaround is nuisance but is somewhat reasonable.

For the environment case, the workaround currently is for users to use an export in the command rather than the environment variable section - which is just annoying...

- run:
    command: |
      export PATH=/go/bin:$PATH
      go-vet ....

# instead of
- run:
    command: go-vet ...
    environment:
      PATH=/go/bin:$PATH

1.0 uses the `export` syntax for setting environment variables. This may be sufficient in 2.0 - but it is a bit verbose.

  • Alexey Klochay
  • Nov 23 2017
  • Taking votes
  • Attach files
  • Timothy Clarke commented
    09 Jul 07:59

    To an extent you can already do this

    echo "export working_dir=\"/go/src/github.com/${ORGNAME}/${REPONAME}\""  >> $BASH_ENV

    If you're using bash CircleCI pick this up at each run step. If you want to use it in the current run step you'll need to do source ${BASH_EMV}

    You can't interpolate variables at execution time eg changing ${ORGNAME} and expect ${working_dir} to be different, but that is a limitation of bash

  • Nathan Dintenfass commented
    December 05, 2019 23:00

    Some of the scenarios for this are now covered by pipeline variables.  See: https://circleci.com/docs/2.0/pipeline-variables/#pipeline-values

  • andrew batz commented
    March 22, 2019 15:13

    There hast to be a way to write to circle variables, parameters, etc without changing the config.yml 

    This is an implementation idea that would be super convenient, but I think it's valuable to point out the underlying need and that it's still applicable. 

  • Isaac Seymour commented
    June 04, 2018 12:11

    Would this also apply to cache keys? e.g., we extensively use YAML references to deduplicate build steps that check asset generation in every locale, so it'd be great to be able to put
    ```yaml
    restore_cache:

      keys:

        - assets-v1-{{ .environment.COUNTRY }}-{{ .Branch }}-{{ .Version }}
        - ...
    ```
    in a reference, and then be able to run that job multiple times for different values of `$COUNTRY`.

  • Nathan Dintenfass commented
    April 10, 2018 19:37

    This issue is on our radar -- we are likely to introduce first-class parameters to jobs along with reusable commands that would make it easier to make values like cache paths, working directory, images, and others dynamic. We are, though, hesitant to do this with env vars, as our general approach is to make env var injection as "late" in the process as possible and only in a runtime environment you have specified. Config processing is done before your code is executed and before the runtime environment you specify in your executor is provisioned, and we do not generally want to be accessing your env vars at that stage. 

  • Mike Kobit commented
    April 10, 2018 18:29

    -1 from me - I think making interpolation in YAML makes each step, or certain steps, or the file, or certain attributes on steps, or any other location it can be used confusing. Adding more interpolation features and special processing of the YAML static data will make it more difficult write, debug, and comprehend. I think a much better idea is https://circleci.com/ideas/?idea=CCI-I-184 which would allow for users to do this themselves in their own complex world without making this the common case for all users. Adding even more special processing to YAML can end up like GitLab's absolutely bonkers include directive (https://docs.gitlab.com/ee/ci/yaml/#include

  • Guest commented
    March 12, 2018 20:48

    env interpolation needs to be supported throughout config.yml, not just in specific sections.  it's essential for numerous workflow use cases.  

  • Mike Schinkel commented
    March 06, 2018 22:57

    I have set up an open-source solution I call WP DevOps specifically for managing WordPress sites although it is not yet v1.0 and as such does not yet have an example circle.yml file.

    Given WP DevOps architecture it would be utterly impossible to move it to 2.0 without environment var interpolation

  • Dominik GM commented
    February 14, 2018 15:47

    Environment interpolation is a long requested feature: https://discuss.circleci.com/t/reference-an-environment-variable-in-environment-yaml-map/10792

    @Alexey: What is the workaround for the `working_directory`?

  • Jamie Talbot commented
    February 05, 2018 06:18

    (I tried creating a fixed symlink at /tmp/node_modules pointing to the dynamic directory, but it unfortunately caches the symlink, so the contents of the directory it points to.)

     

    Likewise in the checksum of the cache key. Current workaround is to cat the contents of a dynamic file to a static `.lock` file. This appears to work, but the solution as a whole doesn't work because I can't save a dynamic path.

  • Jamie Talbot commented
    February 05, 2018 06:06

    I would particularly like to be able to interpolate in `save_cache: paths`, so I can generalise my build steps and interpolate the service name.