Wiki

Clone wiki

linode_jumpbox_config / client_connect_cluster

part of the linode_jumpbox_config project

summary

This page records the procedure required to connect from a client device to the target cluster using an F5VPN client tunneled through an OpenVPN client. The following presumes that you have already successfully accomplished prior steps in this project, i.e.,

  1. correctly setup your jumpbox with proper

    1. base install
    2. OpenVPN install and configuration
  2. correctly setup your client device with proper

    1. F5VPN client install and configuration
    2. OpenVPN client install and configuration

overview

To a first approximation, the process of connecting a client device (once fully installed/configured) to the cluster is

  1. Establish initial conditions: e.g., check/set local routes, determine local public and private IP#s.

  2. Start the OpenVPN client. (This also resets the client's public IP#.)

  3. Start the F5VPN client. This requires

    1. Login to the F5VPN website. (This requires the IP# acquired via the OpenVPN.)
    2. Start the F5VPN client (using the F5VPN website's UI).
  4. Correct the routes set by the F5VPN client.

  5. ssh to a cluster login node.

The process is described/guided by client_networking_investigation.txt in much more detail: the intention is that the user can

  1. Download client_networking_investigation.txt locally.

  2. Edit a few "project metaconstants": these are environment variables near the top of the local client_networking_investigation.txt (using bash syntax, or translate to new environment) which point to needed code and properties files:

    • PROJECT_DIR: must match the path to which you have git cloned this project. E.g., ${HOME}/code/projects/linode_jumpbox_config
    • SCRIPTS_DIR: default value= ${PROJECT_DIR}/scripts must be the directory/folder containing this project's properties and scripts (et al.)
    • PRIVATE_PROPERTIES_FP: default value= ${SCRIPTS_DIR}/private.properties must match the path to your previously-created and -edited private properties.
    • PUBLIC_PROPERTIES_FP: default value= ${SCRIPTS_DIR}/public.properties must match the path to your previously-created and -edited public properties.
  3. Cut'n'paste relevant text from the client_networking_investigation.txt into a bash terminal.

procedure

The following is intended merely to describe steps taken in client_networking_investigation.txt, which should be regarded as authoritative.

establish initial state

client device

  1. Set metaconstants described above in your shell's environment.

  2. Check basic networking with network_test.sh . If needed,

  3. Get INITIAL_PUBLIC_IPN with get_public_IP_n.sh

  4. Ensure that local OpenVPN client is !running, with kill_OpenVPN_client.sh

  5. Check that there is no firewall running on the client device with sudo iptables -L

  6. Get information about local/non-public addresses (e.g., LOCAL_IPN) and routes with get_local_routes.sh

iptables

sudo iptables -L on the client device should produce output like

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
routes

After restarting the client device, when connected to my ISP's modem, ip route shows the following routes on the client device (IP# parameterization and route numbering added for clarity):

0: default via 192.168.1.1 dev eth0 proto static
1: 169.254.0.0/16 dev eth0 scope link metric 1000
2: 192.168.1.0/24 dev eth0 proto kernel scope link src LOCAL_IPN

E.g., in the above route#=2, LOCAL_IPN == 192.168.1.142 == the laptop's LAN IP# from ip addr show dev eth0 | fgrep -e 'inet '

jumpbox

  1. Ensure remote OpenVPN server is running on the jumpbox. This can be done from the client device with `check_OpenVPN_server.sh`_.

  2. Check the firewall on the jumpbox: ssh to it and run sudo iptables -L

  3. Check routes on the jumpbox. This can be done from the client device with (TODO: script to ensure properties load):

    ssh ${MYNODE_USER_NAME}@${MYNODE_IPV4} 'ip route show' 2> /dev/null
    # see example routes below
    
iptables

Output from sudo iptables -L on the jumpbox should (aside from the fail2ban fragment) be very similar to this text:

Chain INPUT (policy ACCEPT)
target     prot opt source                  destination
fail2ban-ssh  tcp  --  anywhere             anywhere      multiport dports ssh
ACCEPT     all  --  anywhere                anywhere
ACCEPT     all  --  anywhere                anywhere

Chain FORWARD (policy ACCEPT)
target     prot opt source                  destination
ACCEPT     all  --  anywhere                anywhere      state RELATED,ESTABLISHED
ACCEPT     all  --  OPENVPN_DOMAIN_IPN/24   anywhere
REJECT     all  --  anywhere                anywhere      reject-with icmp-port-unreachable
ACCEPT     all  --  anywhere                anywhere
ACCEPT     all  --  anywhere                anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source                  destination

Chain fail2ban-ssh (1 references)
target     prot opt source                  destination
DROP       all  -- fixed-189-104-103.iusacell.net  anywhere
RETURN     all  --  anywhere                anywhere

where OPENVPN_DOMAIN_IPN is the OpenVPN's "domain IP#" (currently OPENVPN_DOMAIN_IPN which has not been observed to change).

routes

On the jumpbox, ip route shows (with IP# parameterization and route numbering added for clarity):

0: default via JUMPBOX_GATEWAY_IPN dev eth0
1: OPENVPN_DOMAIN_IPN/24 via 10.8.0.2 dev tun0
2: 10.8.0.2 dev tun0  proto kernel  scope link  src OPENVPN_GATEWAY_IPN
3: JUMPBOX_DOMAIN_IPN/24 dev eth0  proto kernel  scope link  src JUMPBOX_PUBLIC_IPN

where

  • JUMPBOX_PUBLIC_IPN is the jumpbox's public IP#, as shown by (e.g.) get_public_IP_n.sh
  • JUMPBOX_GATEWAY_IPN is ${JUMPBOX_PUBLIC_IPN%.*}.1
  • JUMPBOX_DOMAIN_IPN is ${JUMPBOX_PUBLIC_IPN%.*}.0
  • OPENVPN_DOMAIN_IPN is the OpenVPN's "domain IP#"== (currently 10.8.0.0 which has not been observed to change)
  • OPENVPN_GATEWAY_IPN is the OpenVPN's "gateway IP#"== ${OPENVPN_DOMAIN_IPN%.*}.1 (currently 10.8.0.1 which has not been observed to change)

start OpenVPN client

On the client device,

  1. Run start_OpenVPN_client.sh
  2. Get JUMPBOX_PUBLIC_IPN with get_public_IP_n.sh
  3. Get information about OpenVPN-related addresses and routes (on the client device) with get_OpenVPN_routes.sh
  4. If desired, interrogate post-OpenVPN-activation state in more detail

routes

After starting the OpenVPN client on the client device, ip route shows the following routes on the client device (IP# parameterization and route numbering added for clarity):

0: 0.0.0.0/1 via OPENVPN_ENDPT_IPN dev tun0
1: default via 192.168.1.1 dev eth0  proto static
2: OPENVPN_GATEWAY_IPN via OPENVPN_ENDPT_IPN dev tun0
3: OPENVPN_ENDPT_IPN dev tun0  proto kernel  scope link  src OPENVPN_SRCPT_IPN
4: 128.0.0.0/1 via OPENVPN_ENDPT_IPN dev tun0
5: 169.254.0.0/16 dev eth0  scope link  metric 1000
6: JUMPBOX_PUBLIC_IPN via 192.168.1.1 dev eth0
7: 192.168.1.0/24 dev eth0  proto kernel  scope link  src LOCAL_IPN

where

  • JUMPBOX_PUBLIC_IPN is the jumpbox's public IP#, as shown by (e.g.) get_public_IP_n.sh
  • LOCAL_IPN is the client device's LAN IP# , as shown by (e.g.) ip addr show dev eth0 | fgrep -e 'inet '
  • OPENVPN_ENDPT_IPN is "the server side of the pipe" (currently 10.8.0.5, which has not been observed to change)
  • OPENVPN_GATEWAY_IPN is the OpenVPN's "gateway IP#" (currently 10.8.0.1, which has not been observed to change)
  • OPENVPN_SRCPT_IPN is "the client side of the pipe" (currently 10.8.0.6, which has not been observed to change)

start F5VPN client

  1. Start F5NAP'ed browser
  2. Browse to remote-access website
  3. Login to remote-access website
  4. If desired, interrogate post-login state in more detail (should still match post-OpenVPN-activation state)
  5. Start F5VPN using remote-access website UI

routes

After starting the F5VPN client on the client device, ip route shows the following routes on the client device (IP# parameterization and route numbering added for clarity):

0: 0.0.0.0/1 via F5VPN_ENDPT_IPN dev ppp0  proto none  metric 1
1: default via 192.168.1.1 dev eth0  proto static
2: F5VPN_GATEWAY_IPN dev ppp0  proto kernel  scope link  src F5VPN_ENDPT_IPN
3: 128.0.0.0/1 via F5VPN_ENDPT_IPN dev ppp0  proto none  metric 1
4: F5VPN_PUBLIC_IPN via OPENVPN_ENDPT_IPN dev tun0  proto none  metric 1

where

  • F5VPN_PUBLIC_IPN seems to be the public IP# of whatever the client device has now connected to (though that is merely my guess). It seems to change with every connection.
  • LOCAL_IPN is the client device's LAN IP# , as shown by (e.g.) ip addr show dev eth0 | fgrep -e 'inet '
  • F5VPN_ENDPT_IPN seems to be "the server side of the pipe" to which the the client device has now connected (though that is merely my guess). It also seems to change with every connection, but always seems to begin with 10.144..
  • F5VPN_GATEWAY_IPN seems to be the "gateway IP#" for the F5VPN (currently 10.144.0.1, which has not been observed to change)
  • OPENVPN_ENDPT_IPN is "the server side" of the OpenVPN (currently 10.8.0.5, which has not been observed to change)

problem

The above routeset is broken, in that it does not appropriately route traffic from the client device through the {jumpbox, OpenVPN server}. Hence network_test.sh shows the following problems:

  1. ping:

    > network_test.sh: ping -c 4 162.220.62.158
    > PING 162.220.62.158 (162.220.62.158) 56(84) bytes of data.
    >
    > --- 162.220.62.158 ping statistics ---
    > 4 packets transmitted, 0 received, 100% packet loss, time 3024ms
    
  2. DNS:

    > network_test.sh: nslookup api.hostip.info
    > ;; connection timed out; no servers could be reached
    

Note that

  • 162.220.62.158 and api.hostip.info are defined public properties
  • 162.220.62.158 is the IP# corresponding to api.hostip.info (and should be returned when DNS is working)

reconfigure F5VPN

Note the following must currently be done quickly, since the current networking problem will quickly crash the current network configuration.

client device

  1. If desired, interrogate post-F5VPN-activation state in more detail
  2. Get information about F5VPN-related addresses and routes with get_F5VPN_routes.sh
  3. Delete F5VPN-misconfigured routes with delete_current_routes.sh
  4. Set desired routes on F5VPN with set_F5VPN_routes.sh

set_F5VPN_routes.sh currently attempts to set the following routes on the client device:

192.168.1.0/24 dev eth0  proto kernel  scope link  src LOCAL_IPN
0.0.0.0/1 via F5VPN_ENDPT_IPN dev ppp0  proto none  metric 1
default via 192.168.1.1 dev eth0  proto static
F5VPN_GATEWAY_IPN dev ppp0  proto kernel  scope link  src F5VPN_ENDPT_IPN
128.0.0.0/1 via F5VPN_ENDPT_IPN dev ppp0  proto none  metric 1
JUMPBOX_PUBLIC_IPN via 192.168.1.1 dev eth0
F5VPN_PUBLIC_IPN via OPENVPN_ENDPT_IPN dev tun0  proto none  metric 1
F5VPN_GATEWAY_IPN via OPENVPN_GATEWAY_IPN dev tun0

using the following identities:

  • LOCAL_IPN is the client device's original/LAN IP#
  • JUMPBOX_PUBLIC_IPN is (currently, but this should be stable) the jumpbox's public IP#
  • OPENVPN_ENDPT_IPN is the OpenVPN's "endpoint IP#" (currently 10.8.0.5 and has not been observed to change)
  • OPENVPN_GATEWAY_IPN is the OPENVPN's gateway IP# (i.e., ${OPENVPN_ENDPT_IPN%.*}.1)
  • F5VPN_ENDPT_IPN is the F5VPN's "endpoint IP#" (10.144.14.186 in this example--may change between connections?)
  • F5VPN_GATEWAY_IPN is the F5VPN's gateway IP# (i.e., ${F5VPN_ENDPT_IPN%.*}.1)
  • F5VPN_PUBLIC_IPN is the F5VPN server's public IP# (which is known to change on each connection)

(Note also that ip route add will not set routes containing proto none, so I strip that text wherever it occurs.)

use F5VPN

  1. Interrogate post-F5VPN-activation state with, e.g.,

    • ip route show
    • ip link show
    • ip addr show
    • network_test.sh
  2. Start ssh session on cluster login node

close F5VPN

  1. End all cluster ssh sessions
  2. Disconnect from F5VPN (using the F5VPN website's UI)
  3. Logout from remote-access website
  4. If desired, interrogate post-F5VPN-deactivation state

stop OpenVPN client

  1. Disconnect from OpenVPN with kill_OpenVPN_client.sh

  2. If desired, interrogate post-OpenVPN-deactivation state

  3. If needed,

F5VPN problem

Everything looks good until I attempt to reconfigure the F5VPN on the client device:

me@it:~$ source ${SCRIPTS_DIR}/set_F5VPN_routes.sh
> ip_route_functions.sh: about to set F5VPN route#=0: '192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.142'
> ip_route_functions.sh: sudo ip route add 192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.142

> ip_route_functions.sh: about to set F5VPN route#=1: '0.0.0.0/1 via 10.144.12.214 dev ppp0    metric 1'
> ip_route_functions.sh: sudo ip route add 0.0.0.0/1 via 10.144.12.214 dev ppp0    metric 1

> ip_route_functions.sh: about to set F5VPN route#=2: 'default via 192.168.1.1 dev eth0  proto static'
> ip_route_functions.sh: sudo ip route add default via 192.168.1.1 dev eth0  proto static

> ip_route_functions.sh: about to set F5VPN route#=3: '10.144.0.1 dev ppp0  proto kernel  scope link  src 10.144.12.214'
> ip_route_functions.sh: sudo ip route add 10.144.0.1 dev ppp0  proto kernel  scope link  src 10.144.12.214

> ip_route_functions.sh: about to set F5VPN route#=4: '128.0.0.0/1 via 10.144.12.214 dev ppp0    metric 1'
> ip_route_functions.sh: sudo ip route add 128.0.0.0/1 via 10.144.12.214 dev ppp0    metric 1

> ip_route_functions.sh: about to set F5VPN route#=5: '173.230.129.18 via 192.168.1.1 dev eth0'
> ip_route_functions.sh: sudo ip route add 173.230.129.18 via 192.168.1.1 dev eth0

> ip_route_functions.sh: about to set F5VPN route#=6: '134.67.15.30 via 10.8.0.5 dev tun0    metric 1'
> ip_route_functions.sh: sudo ip route add 134.67.15.30 via 10.8.0.5 dev tun0    metric 1
> RTNETLINK answers: Network is unreachable

> ip_route_functions.sh: about to set F5VPN route#=7: '10.144.0.1 via 10.8.0.1 dev tun0'
> ip_route_functions.sh: sudo ip route add 10.144.0.1 via 10.8.0.1 dev tun0
> RTNETLINK answers: Network is unreachable

> ip_route_functions.sh: date
> Thu Mar 19 23:03:10 EDT 2015
> ip_route_functions.sh: ip route show
> 0.0.0.0/1 via 10.144.12.214 dev ppp0  metric 1
> default via 192.168.1.1 dev eth0  proto static
> 10.144.0.1 dev ppp0  proto kernel  scope link  src 10.144.12.214
> 128.0.0.0/1 via 10.144.12.214 dev ppp0  metric 1
> 173.230.129.18 via 192.168.1.1 dev eth0
> 192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.142

Resulting networking fails:

  1. ping:

    > network_test.sh: ping -c 4 162.220.62.158
    > PING 162.220.62.158 (162.220.62.158) 56(84) bytes of data.
    >
    > --- 162.220.62.158 ping statistics ---
    > 4 packets transmitted, 0 received, 100% packet loss, time 3024ms
    
  2. DNS:

    > network_test.sh: nslookup api.hostip.info
    > ;; connection timed out; no servers could be reached
    
  3. ssh:

    $ ssh [...]
    ssh: Could not resolve hostname <login host/>: Name or service not known
    

Updated