Secured Environment variable "HEROKU_API_KEY" prevent Pipelines from working properly

Issue #15082 invalid
Gabriel Marcolino
staff created an issue

When we try to use Bitbucket Pipelines to deploy code to Heroku we face the following error:

Deploying Heroku Version 123456789012345678901234567890123456123
Traceback (most recent call last):
File "<string>", line 1, in <module>
KeyError: 'source_blob'
Traceback (most recent call last):
File "<string>", line 1, in <module>
KeyError: 'source_blob'
curl: no URL specified!
curl: try 'curl --help' or 'curl --manual' for more information
Traceback (most recent call last):
File "<string>", line 1, in <module>
KeyError: 'output_stream_url'
curl: try 'curl --help' or 'curl --manual' for more information

Looking into StackOverflow threads, we found that the cause may be the Secured Environment variable, the solution proposed there is to store it in plain text, which is not the best option since it exposes the secret KEY.

Comments (4)

  1. Matt Ryall staff

    This seems like an escaping bug in the URL being generated by the Heroku deployment script, not a bug in Pipelines.

    @Gabriel Marcolino - can you provide any more detail on what command the customer is running? Do you have a link to the support case?

  2. Gabriel Marcolino staff reporter

    Hi @Matt Ryall,

    These are the commands that they are running:

    #!/bin/bash
    git archive --format=tar.gz -o deploy.tgz $BITBUCKET_COMMIT
    
    HEROKU_VERSION=$BITBUCKET_COMMIT
    APP_NAME=$HEROKU_APP_NAME # Your app's name in heroku goes here
    
    echo "Deploying Heroku Version $HEROKU_VERSION"
    
    URL_BLOB=`curl -s -n -X POST https://api.heroku.com/apps/$APP_NAME/sources \\\ -H 'Accept: application/vnd.heroku+json; version=3' \\\ -H "Authorization: Bearer $HEROKU_API_KEY"`
    
    PUT_URL=`echo $URL_BLOB | python -c 'import sys, json; print(json.load(sys.stdin)["source_blob"]["put_url"])'`
    GET_URL=`echo $URL_BLOB | python -c 'import sys, json; print(json.load(sys.stdin)["source_blob"]["get_url"])'`
    
    curl $PUT_URL -X PUT -H 'Content-Type:' --data-binary @deploy.tgz
    
    REQ_DATA="{\"source_blob\": {\"url\":\"$GET_URL\", \"version\": \"$HEROKU_VERSION\"}}"
    
    BUILD_OUTPUT=`curl -s -n -X POST https://api.heroku.com/apps/$APP_NAME/builds \\\ -d "$REQ_DATA" \\\ -H 'Accept: application/vnd.heroku+json; version=3' \\\ -H "Content-Type: application/json" \\\ -H "Authorization: Bearer $HEROKU_API_KEY"`
    
    STREAM_URL=`echo $BUILD_OUTPUT | python -c 'import sys, json; print(json.load(sys.stdin)["output_stream_url"])'`
    
    curl $STREAM_URL
    
  3. Matt Ryall staff

    Looking at the first errors messages presented above, I can deduce some details of the problem:

    Traceback (most recent call last):
    File "<string>", line 1, in <module>
    KeyError: 'source_blob'
    Traceback (most recent call last):
    File "<string>", line 1, in <module>
    KeyError: 'source_blob'
    

    These indicate that both PUT_URL and GET_URL lines fail because the data in URL_BLOB does not contain a source_blob attribute. This means the response from Heroku to the POST request to api.heroku.com/apps/$APP_NAME/sources wasn't an expected response, and was probably an error response.

    You should add a line to echo $URL_BLOB immediately after it is set, and check the result of this call to Heroku. This response should indicate what the problem is.

    The quoting in the script looks fine, and secure variables in Pipelines are not broken, so there must be some other issue. Perhaps the username+password is no longer correct, or is not correctly encoded. Perhaps it contains a special character (like &) that is being interpreted by the shell.

    If the error seems unintuitive, my next step would be running the commands locally (including with environment variables set to the correct values) to check that they work properly in a shell.

    I'll close this ticket as invalid because despite that StackOverflow post, this is a not a reproducible bug. There is no difference between secure and insecure variables as far as the build script is concerned - both are exposed as environment variables. But please feel free to reply with further questions if you can't get it working.

  4. Log in to comment