Mask secrets in logs

Secrets may be accidentally printed in the build logs. Travis hide them in the logs masking them as [secret]  every time that it finds a match with the provided secrets. See https://discuss.circleci.com/t/masking-secrets-in-output-logs/22231

  • Guest
  • May 8 2018
  • Shipped
  • Sep 9, 2019

    Admin Response

    Hi All,

    We're currently beta testing thris feature. If you'd like to sign up to get early access, you can do so here https://forms.gle/e1ucYVaLZJ83Dvw76 . Otherwise, you should see this be automatically enabled over the next few weeks.

    Best,

    George

  • Attach files
  • Buck DeFore commented
    May 21, 2018 18:06

    ran into this today and exposed an aws secret key, expecting behavior of other ci systems. bitbucket pipelines for example protects against this.

  • Bill Wang commented
    July 31, 2018 01:40

    I do think the develop team at Circle CI should promote this feature as highest priority for enterprise (pay) users. 

    I tested with other pipelines, such as bamboo, travis ci, bitbucket pipeline, no secrets can be echo in build logs. 

    This feature becomes a block when we plan to move our application to production environment, because the developers can easily get the api keys (such as aws, database, apigee, etc) after change the pipeline from the feature branches. 

  • Paul Jolly commented
    August 08, 2018 18:13

    I've voted for this, but just to add more colour to my vote. Because the reason for this proposal is more than just "write sensible scripts", because it's entirely possible code you are testing also accidentally leaks secrets through its test output. And you could keep going enumerating the different paths through which your secret environment variables get accidentally exposed.

     

    Having the CI system as a backstop, replacing secret values in _all_ output, just seems like the right thing to do therefore. 

  • Greg Zapp commented
    August 12, 2018 04:04

    This would absolutely be essential for us.

     

    Really hoping to try out Circle as a Travis replacement however I'm contemplating all the extra adjustments I'll need to make. Currently lean pretty heavily on their secret scrubbing, and I like a good set -x in my builds..

    After the adjustments there would be the extra safe guard considerations as an accidental leak would probably go unnoticed and be very bad.

  • Graham Williamson commented
    August 18, 2018 05:38

    Bitbucket mask the "secret" variables with the variable name. This was actually confusing at first, until I realised what was occurring.  I like a combination of both travis and bitbucket. So, after defining an AWS_SECRET_KEY in the environment variables and checking a box that it's a secret, it would be good to mask with something like [PROJECT-SECRET:::AWS_SECRET_KEY].  If I had several secrets and something was going wrong, I'd want to know which secret was which if I was debugging. Also know where the secret came from would be good, project or org/context etc.

  • Peter Galkaev commented
    November 14, 2018 11:36

    From CircleCI docs about setting environment variables:

    The value of the variables are neither readable nor editable in the app after they are set.

    Well... they are readable. How hard is it to fix this bug?

  • Dmitry Rozhdestvenskiy commented
    December 12, 2018 14:39

    Accidentally used `env` while debugging, can't find any plausible way to clear build log. Definitely should be masked.

  • Vusal Khalilov commented
    December 18, 2018 13:37

    Many companies  can't use this service because of absence of this feature. Organizations with lots of new engineers can't protect themselves against possible threats that printing of environment variables may cause. 

  • Yasmany Cubela Medina commented
    February 20, 2019 02:09

    important secrets are been displayed on logs, like on api call to trigger new circle jobs

  • Anthony Yeong commented
    March 05, 2019 03:06

    This is a must have feature.

  • Christer Edvartsen commented
    March 05, 2019 16:34

    This feature should absolutely be prioritized imho.

  • Alexandre Kaskasoli commented
    April 26, 2019 16:23

    I'm a bit flabbergasted that this isn't already a thing for CircleCI.

  • Piotr S commented
    May 09, 2019 14:00

    IMO it should depend on the configuration because sometimes I want to check whether I passed a correct env.

  • George Reyes commented
    May 10, 2019 18:22

    We agree that masking the variables in the log output would be a very valuable feature for CircleCI to have. However, the work is currently behind other high-urgency items that are necessary to maintain the stability of our platform. We will consider this feature again in the future.

  • Yasmany Cubela Medina commented
    May 10, 2019 18:39

    And how should we protect in the meanwhile? this secrets are suppose to be secrets for a reason. IMHO this is a highest security priority

  • Julie Magee commented
    June 18, 2019 10:33
                   
                   

    https://circleci.circleci.com/attachments?attachment_only=true&selected_id=6703816136338172082&single_viewer=false

  • Valentin Hervieu commented
    August 13, 2019 05:31

    This is a really important issue. See this article for example: https://www.zdnet.com/google-amp/article/ci-build-logs-continue-to-expose-company-secrets

     

    I cannot opensource a project of mine because of this issue. In there, I'm using now.sh to deploy my application and the token is exposed (https://github.com/zeit/now-cli/issues/1937).

  • Haikal Pribadi commented
    September 13, 2019 11:05

    Are some users randomly picked to be tested on this beta feature? Cause I think we are being affected by it, and it's incorrectly masking information that we need to see in the logs!!!!! We submitted an issue ticket yesterday and no one has responded so far :(

  • Yvan commented
    November 25, 2019 12:23

    It seems that we have an ENV var set in our `config.yml` wich is `RAILS_ENV=test`. And the Circle CI masks very occurrences of `test` in our CI output. I assume this, because the documentation is not clear about what exactly is masked and how to add/remove a masked term.

    So, copy tasting a test file path from Circle CI is now broken and has to be fixed manually.

  • Yvan commented
    November 25, 2019 13:13

    Circle CI currently masks every occurence of ENV variables value. In our project, we have `RAILS_ENV: test` set in config.yml resulting in all occurrences of `test` being masked from our build output.

    Our test files path looks like `test/models/xxxxxx_test.rb`, so these kind of paths now then become `****/models/xxxxxx_****.rb` making copy pasting the path more difficult and confusing, especially for beginners in our team.

    Could you add some sort of whitelist, or even better mask only env vars set in Circle CI Admin UI and not what is in `config.yml`, since it is a file in a repo, and there is no reason to write secrets there as good practices says that repos should not contain any secret.

  • Erwan Baynaud commented
    28 Jan 18:55

    A whitelist should be implemented!