Pipelines - id_rsa not found since this morning (Deployment process broken)

Issue #14729 resolved
Robin Rijkeboer
created an issue

Since today, whenever I run a pipelines build this happens:

scp build.tar.gz user@$ACC_SERVER_IP:~/
no such identity: /opt/atlassian/pipelines/agent/data/id_rsa: No such file or directory
Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,password).

Pipelines are an essential part in our deployment proccess and we seem to hit this error on all our pipelines deploys.

Here is a community thread where people also encountered this issue since this morning: https://community.atlassian.com/t5/Bitbucket-questions/Could-opt-atlassian-pipelines-agent-data-id-rsa-become-invalid/qaq-p/631206#U631240

Official response

  • Samuel Tannous staff

    Hi everyone,

    We've spent the past couple of days investigating how the original change could have caused this issue. Thank you to the customers who assisted in this investigation. Unfortunately we were unable to reproduce the problem including in any of the customer repositories who experienced the issue and granted us permission to debug directly.

    The original code change was intended to enable SSH key support for non-root users. We have rearchitected the solution to reduce the risk to existing configurations but as we were unable to identify root cause there remains a small chance that this change may affect some users. We are prepared to work with those users and revert the change if necessary.

    Regards Sam

Comments (50)

  1. James Quinton

    We're having the same issue.

    No change to our keys, however we also get the error:

    no such identity: /opt/atlassian/pipelines/agent/data/id_rsa: No such file or directory

  2. Ronald Chia staff

    As a workaround while our team is investigating this issue, you can try adding the following steps in your script, which load the SSH-key to ssh-agent, and you should then be able to run your builds

              - eval `ssh-agent`
              - ssh-add /opt/atlassian/pipelines/agent/data/id_rsa
    
  3. Matthew Drouin

    Doesn't work for us.

    + ssh-add /opt/atlassian/pipelines/agent/data/id_rsa
    /opt/atlassian/pipelines/agent/data/id_rsa: No such file or directory
    

    Is there error we get.

  4. Matthew Drouin

    I created a release script that did nothing but ls -alh /opt/atlassian/pipelines/agent/data and I noticed the file was there @Ronald Chia from support emailed me the suggestion. I moved the recommended solution to the top of my scripts, first things that gets in the script tag, and everything seems to work now.

    I was running it after all my tests and builds previously and that seems to have been causing issues.

  5. Philip Hodder staff

    Hi everyone,

    We've deployed a fix for this issue and are investigating if this is fully resolved.

    Can you please let us know if the issue is resolved or still occurring for you.

    Thanks, Phil

  6. Tim Brandin

    We're still having issues, but it might be a new one, this time it has to do with Host key verification.

    The key exists in the container, I can verify that and I believe it is the right one, and that key is associated with a user with access to the repo we are trying to merge into, but still I get: "Host key verification failed.". And I've tested to add bitbucket in known hosts (even though it says one should need to ad bitbucket in the docs) without any luck.

  7. Tim Brandin

    I've gotten help to resolve our issue with Host key verification, somehow the change yesterday made it so that known_hosts was not available anymore to the user we use in our docker image, which means we have to build up the known_hosts ourself and cannot rely on the interface in bitbucket pipelines ssh key settings.

  8. Samuel Tannous staff

    Hi everyone,

    We've spent the past couple of days investigating how the original change could have caused this issue. Thank you to the customers who assisted in this investigation. Unfortunately we were unable to reproduce the problem including in any of the customer repositories who experienced the issue and granted us permission to debug directly.

    The original code change was intended to enable SSH key support for non-root users. We have rearchitected the solution to reduce the risk to existing configurations but as we were unable to identify root cause there remains a small chance that this change may affect some users. We are prepared to work with those users and revert the change if necessary.

    Regards Sam

  9. Log in to comment