A usage scenario:
In my case, tunneling from a server through remote gateway(s) which handles the routing for a remote network consisting of embedded linux nodes, which are bridged together in strings of a daisy chain topology using STP, linked by fiber backbone of switches down to the gateways. In case you’re wondering, the nodes span across a bridge.
The two gateways are different networks with 2 separate internet up links. So it’s fair to say I’m dealing with a very unconventional network, that I have to make work.
The reason ansible was chosen for automating these nodes is because the aspect of being able to use it both for DevOps ansible-playbook stuff, and also as a substitute for parallel-ssh since I’m a sysadmin. This makes it easier to maintain the large scale I’m dealing with by simply dividing up separate networked sections into inventory child groups.
The setup and maintenance of the server is also deployed by ansible, as we thought it fitting to use the same tools for the server side of our infrastructure as we use for the embedded, but to manually configure this is the general approach.
…
Consider the following:
Node Control Server(Ansible + OpenVPN) —> Gateways —> Nodes
On server:
Set your ansible cfg to use ‘ssh’ as transport.
Ansible.cfg:
[defaults]
# some basic default values…
hostfile = /home/user/ansible/hosts-all
library = /usr/share/ansible
remote_tmp = $HOME/.ansible/tmp
pattern = *
forks = 10
poll_interval = 15
sudo_user = root
#ask_sudo_pass = True
#ask_pass = True
transport = ssh
remote_port = 22
….
[ssh_connection]
# ssh arguments to use
# Leaving off ControlPersist will result in poor performance, so use
# paramiko on older platforms rather than removing it
# SSH arguments tweaked to work better with my network
ssh_args = -o ControlMaster=no -o ServerAliveInterval=30 -o ServerAliveCountMax=4
# if True, make ansible use scp if the connection type is ssh
# (default is sftp)
scp_if_ssh = True
——————————————————————-
In my case I didn’t want strict host key checking, because that was already done on the gateway level, and the embedded linux nodes I’m managing don’t have user space and have the same key settings (so it’s logging in as root anyway). Worth while mentioning that in my use case the gateway link were VPN tunnels.
/etc/ssh_config or ~/.ssh/config
Host *.firstdomain
ProxyCommand ssh user@gateway-hostname1 nc %h %p
IdentityFile ~/.ssh/id_rsa_gw1
StrictHostKeyChecking=no
User root
Host *.seconddomain
ProxyCommand ssh user@gateway-hostname2 nc %h %p
IdentityFile ~/.ssh/id_rsa_gw2
StrictHostKeyChecking=no
User root
——————————————————————-
Then create your ansible inventory file as such:
ansibleuser@node-control-server:~$ ansible/hosts-all
[production:children]
childgroup1
childgroup2
childgroup3
childgroup4
….
[childgroup1]
node124 ansible_ssh_host=node124.firstdomain
node128 ansible_ssh_host=node128.firstdomain
node129 ansible_ssh_host=node129.firstdomain
node130 ansible_ssh_host=node130.firstdomain
node113 ansible_ssh_host=node113.firstdomain
….
[childgroup2]
node110 ansible_ssh_host=node110.firstdomain
node119 ansible_ssh_host=node119.firstdomain
node133 ansible_ssh_host=node133.firstdomain
node117 ansible_ssh_host=node117.firstdomain
node135 ansible_ssh_host=node135.firstdomain
…
[childgroup3]
node582 ansible_ssh_host=node582.seconddomain
node576 ansible_ssh_host=node576.seconddomain
node573 ansible_ssh_host=node573.seconddomain
node580 ansible_ssh_host=node580.seconddomain
node578 ansible_ssh_host=node578.seconddomain
…
[childgroup4]
node567 ansible_ssh_host=node567.seconddomain
node577 ansible_ssh_host=node577.seconddomain
node592 ansible_ssh_host=node592.seconddomain
node561 ansible_ssh_host=node561.seconddomain
node571 ansible_ssh_host=node571.seconddomain
——————————————————————
Gateways:
/etc/hosts should be maintained to handle node to IP resolution as such:
IP Address NodeHostname NodeHostname.<firstdomain|secondomain>.
i.e….
10.0.0.123 node123 node123.firstdomain
Testing:
ansibleuser@node-control:~$ ansible childgroup2 -m ping
node108 | success >> {
“changed”: false,
“ping”: “pong”
}
node116 | success >> {
“changed”: false,
“ping”: “pong”
}
node109 | success >> {
“changed”: false,
“ping”: “pong”
}
node120 | success >> {
“changed”: false,
“ping”: “pong”
}
node118 | success >> {
“changed”: false,
“ping”: “pong”
}
node106 | success >> {
“changed”: false,
“ping”: “pong”
}
node114 | success >> {
“changed”: false,
“ping”: “pong”
}
node115 | success >> {
“changed”: false,
“ping”: “pong”
}
… as you can see we have connectivity to the nodes, and the proxying automatically handled via SSH and ansible based on name resolution. You can now scale this over multiple sites by simply adding more gateways to the SSH config file and maintaining them in your ansible inventory of the nodes by hostname.