Accessing Private Repositories

Issue #12795 resolved
Aykut ARAS created an issue

Hi,

I am trying to add an ssh key for installing private git repositories for my build script. This is a requirement for php composer to access other private packages.

First problem occured when rsa fingerprint is not added automatically so I added a script command: - mkdir ~/.ssh - chmod 700 ~/.ssh - echo 'bitbucket.org,131.103.20.168 SOMECHARS' > ~/.ssh/known_hosts

That solved fingerprint problem. After that I tried to added an private key from command line but I need to add it securely. So I tried to add private key via environment variable but env vars removes new lines and that didn't work either. - echo $PRIVATE_KEY > ~/.ssh/private_rsa - chmod 600 ~/.ssh/private_rsa - eval ssh-agent -s - ssh-add ~/.ssh/private_rsa

Is there any way to add a private key or accessing our own private repositories somehow?

Comments (17)

  1. Sten Pittet

    Hi Aykut,

    The workaround that Günther provided is correct and we recommend using HTTPS today to close private repositories.

    Thanks,

    Sten

  2. Aykut ARAS reporter

    I see your point but cloning with https is too troublesome for our day to day use. When one repository depends more private repositories (say 20) it keeps asking password one by one. Instead using ssh solves this problem pretty easily.

    I will try this approach and see how to it goes but it would be great to add ssh capabilities to piplines.

  3. Günther Foidl

    it keeps asking password one by one.

    Why? You can specify the password in the url, like so username:password. For instance git clone --branch="master" --depth 1 https://gfoidl:$API_KEY@bitbucket.org/gfoidl/myRepo.git

    And set $API_KEY as environment variable in the pipelines settings.

  4. Aykut ARAS reporter

    Actually I think this is a desicion how you import private repositories. Currently composer (php package manager) seems to support two approach too. SSH approach (https://getcomposer.org/doc/05-repositories.md#using-private-repositories) and Basic Auth approach (https://getcomposer.org/doc/articles/http-basic-authentication.md).

    I will check for basic auth option but that doesn't seem feasible for us because instead of using users own ssh keys we have to set team deployment key and commit it to source control which seems a bad idea. Handling these keys from env is also a troublesome options.

  5. Günther Foidl

    I can't follow you, maybe I'm missing something.

    Let me explain the setup I used:

    • repo A that should be built by pipelines
    • repo A needs repo B
    • repo B resides in a team account, still I push changes to this repo with my personal account and personal key by ssh. My commit belong to me, and can be identified as commited by me
    • the team account got an api key
    • this api key is set as environment variable in the pipeline settings
    • in the build-scripts for repo A there is no api key or any other credential information. Instead the environment variable for the team account's api key is set. This is safe, because only admins can access this section of the settings.
    • the build-script for repo A clones repo B via https (see above)
    • build succeeds (hopefully)

    because instead of using users own ssh keys we have to set team deployment key and commit it to source control

    You don't have to. Did you mistake api key with team deployment key?

    HTH.

  6. Aykut ARAS reporter

    Ok let me try to make things clear.

    • repo A that should be build by piplines
    • repo A needs private repo B
    • repo A adds this dependency to its composer.json file which should be committed and can be accessible to all develoeprs like one of two ways below:

    Http Basic Auth Way:

    "repositories": [
        {
            "type": "vcs",
            "url":  "https://myteam:$API_KEY@bitbucket.org/myteam/repoB.git"
        }
    ]
    

    SSH Way:

    "repositories": [
        {
            "type": "vcs",
            "url":  "git@bitbucket.org:myteam/repoB.git"
        }
    ]
    
    • This composer.json should be committed to repo in order to take effect of dependency.
    • Also this composer.json file have to be used in developers own machine to get private dependency.

    I understand your point about $API_KEY solution but this solution solves pipeline problem but creates a new problem which is how a developer use this key from their own developer machines.

    Am I getting clear or messing everything more?

    Have a nice day.

  7. Günther Foidl

    Now I got you 😄

    Therefore I only have a suggestion, not a real solution.
    In pipelines add an early step that changes the urls from ssh to https. They are canonical, so it should not be too hard, and affect only the local clone of the build.

    If pipelines can't (or won't) go with ssh, maybe it is possible to perform this rewrite automatically and configurable (in the yml).

    Have a nice day, too.

  8. Aykut ARAS reporter

    Thats a fantastic idea. I will check that. But I will reopen this issue because it does not fix general accessing problem.

    Thank you for your help.

  9. Aykut ARAS reporter
    • changed status to open

    After further discussions adding http basic auth keys to composer.json does not fix general ssh problem. Also adding apiKey to this variables opens a security issue.

  10. Pleun Vanderbauwhede

    The problem I see with using https + APIkey is that (in case of node) each module will need to explicitly have the key set in the package.json. As far as I can see there's no way to pass in an environment variable in the dependency list, so a key change will require updating every consuming module's package.json, which can become a hassle real fast when working with lots of modules in a larger team. I'm not sure if having an ssh > https switch would really be a solution, unless you also rebuild the dependency string (ssh format > https format with key)

    Ideally there would be some kind of hybrid solution that allows devs to use the default authentication methods locally (https or ssh) so that no authorization information has to be stored in the package.json, while automatically transforming the private dependency references to something the CI can deal with, preferably also passing in the necessary env vars :). Package.json preprocessor anyone?

  11. Ben Buchanan

    +1 to needing this.

    As an alternative, could we grant pipelines read access to a repo like it's a bitbucket user?

  12. Joshua Tjhin Account Deactivated

    Hi all,

    Back in March we added support for generating an SSH key in Pipelines which should make it easier to download private dependencies. After generating and copying the public key, you'll need to add the public key to the repository you need access to.

    Link Text

  13. Log in to comment