Allow customers to connect the pipelines builds to their own VPN server

Issue #12753 open
Samuel Tannous staff created an issue

This will allow builds access to private resources for running integration tests, etc.

Comments (28)

  1. Scott Carpenter

    This is a blocker without this feature because Pipelines can't be used for integration testing.

  2. Andrew Goodchild

    Another way to address this feature is to allow customers to run builds on an AWS ECS cluster in the appropriate AWC VPC. See Issue #12748

  3. Amit Sharma

    This is affecting us as we use sshuttle to connect to the same network as our app & db servers. But we can't do that without docker run --privileged option.


    I was also trying to vpn using sshuttle but ran into the problem of needing to run the container in privileged mode. I found a way around this by adding the following to bitbucket-pipelines.yml:

    • ssh -fN -4 -C -D 41337 -o StrictHostKeyChecking=no -i $JUMBOX_KEY ubuntu@$JUMBOX_IP
    • export http_proxy='socks5://localhost:41337'
    • export https_proxy='socks5://localhost:41337'


  5. Marco Pfeiffer

    I managed to pull of ssh over a vpn connection by degrading openconnect to a port forwarder.

          - step:
              name: Deploy
              image: gibby/openconnect
                # connect to vpn
                - bash -c "echo -n '${VPN_PASSWORD}'|openconnect -u${VPN_USER} --passwd-on-stdin --script-tun --script 'ocproxy -L2201:${SSH_HOST}:22' ${VPM_HOST}" &
                - sleep 5 # should be enough wait for the connection to establish
                # connect to servers
                - SSH_SOCKET1="/tmp/sshsocket1"
                - ssh -nNf -o ControlMaster=yes -o ControlPath=${SSH_SOCKET1} -o ControlPersist=3600 -o StrictHostKeyChecking=no -p 2201 user@localhost
                - SSH1="ssh -o ControlPath=${SSH_SOCKET1}"
                - ${SSH1} - echo "do something"
                - rsync -rlvcz -e "${SSH1}" local_foder user@localhost:/remove_folder
  6. Matt Ryall

    We're starting to look at requirements for this feature, so we can work out how much effort it will take and where to put it on the roadmap.

    If you're interested in VPN support in Pipelines, we need to get some details about what kind of VPN connectivity you're after. Here's a Google form with a few quick questions:

    Please include anything you think is relevant to us building this feature. Please don't include any specific network details. We don't need those here. :)


  7. Luke Jacobs

    When this is implemented it'd be nice to have (at the very least) the following traffic routed through the VPN connection:

    Settings -> SSH Keys -> "Fetch"

    Expected Behavior: When fetching the key, it should do so through the VPN so that the connection can be established. Current Behavior: I can not fetch key from a server that requires a specific IP (VPN) to establish a connection.

    Pipelines Builds

    Expected Behavior: The yml file should either have multiple settings (vpn host, user, pass, dir) or a single setting that selects from a list (managed in the settings, similar to environment variables/known hosts/etc).

    In all builds with this configured, it should route all traffic through the VPN connection automatically without having to install and configure things like openconnect as part of the build process.

    Idea: Provide your own, preconfigured, docker images that have everything set up to allow this functionality?

    Current Behavior: Requires that you install and configure your VPN connection during each build and seems to complicate the build process.

    I'm still trying to get this to even work at all using openconnect and basing it off of @Marco_Pfeifer's answer above. In my case, it seems like it's connected but "git ftp init" times out trying to establish the connection, just like I see on my own machine if I'm not connected to the VPN when I try to SSH in.

  8. Luke Jacobs

    FYI, here is what we ended up using for git ftp through a vpn. Hopefully it helps someone while they're waiting for this.

    $S_HOST = your server ip $VPN_HOST = vpn ip

    # connect to vpn
    - bash -c "echo '$VPN_PASS' | openconnect --servercert $S_CERT -vv -u$VPN_USER --interface=tun0 --script-tun -s 'ocproxy -L61234:$S_HOST' $VPN_HOST" &
    - sleep 5
    # update known_hosts
    - echo $S_KNOWN_HOST >> ~/.ssh/known_hosts
    - cat ~/.ssh/known_hosts
    # git ftp 'push' (or 'init' or 'catchup')
    - git ftp push --insecure --user $S_USER --passwd $S_PASS -vv sftp://localhost:61234$S_ROOT
  9. Robert Gonzalez

    +1 Now that were doing CI/CD (trying to) might have to consider github or codeCommit. Can’t build/deploy without vpn access.

  10. Luke Jacobs

    @Robert, there’s workarounds for the time being by using openconnect to connect to the vpn and proxying the build traffic through to it. See Marco and my own examples above. It was tedious figuring out the first time, but once its working, it works and that’s it. It just sits there and you never need to worry about it again unless your build process needs to change.

    But that’s if you don’t want to have to move off the Atlassian platform and you already have all your stuff on it. Otherwise, certainly shop around. 🙂

  11. Robert Gonzalez

    @Luke Jacobs , thanks for replying. I’d love to keep using Bitbucket pipelines. It’s a much easier system to to use and to teach others to use. I’ve tried yours and Marco’s connection and a combination of the two. I get the same error every time. I don’t often deal with VPN connectivity stuff. I just use a config file in a GUI and I’m off to the races.

    The error I receive is:

    XML response has no "auth" node
    Failed to obtain WebVPN cookie

    The connection appears to do the following.:

    • Connect to <ip>/__session_start__/ and receive a Cookie
    • Next it makes another Gt request receives a 302 back which bounces it to https://<ip>/?src=connect.
    • It gets a HTTP 200 and receives the HTML of the login page (http dump flag shows this).

    It’s getting past the SSL handshake thus reaching the server. Yet, always the same error. I’ve been experimenting in docker and tried openconnect 7.x (ubuntu:18.04) and 6.x (debian:8) while trying to get a working connection. From what I see, the server is giving the CLI a web page as a response. In the second step (above) I wonder if it’s even sending the cookie back in it’s second request in order to maintain the connection.

    Anyway, that’s what I’m dealing with. If you have any other suggestions I’d be glad to hear them.

  12. Log in to comment