Ansible: Reboot server in play-book and wait for it come back.

Ansible is a great and simple configuration management and orchestration tool. Some times we require to reboot server and do some tasks once reboot is done. Today I came across such requirement where I have to upgrade our production Ubuntu server from 10.04 to 14.04. I figured some how on unattended upgrades by using following command.

do-release-upgrade -f DistUpgradeViewNonInteractive

The above command will not ask any question when upgrade is happening. The issue is we can not upgrade directly from 10.04 to 14.04. We have to do it in two folds

Ubuntu 10.04 to 12.04

and then

Ubuntu 12.04 to 14.04

But the issue is we have to reboot the machine once upgraded to 12.04. For this Ansible have a solution withwait_for and local_action modules. Below is the snippet from my role which include three task

Task 1: Upgrading Ubuntu 10.04 to 12.04

Task 2: Restart remote machine

Task 3: Wait for the restart of remote machine to complete and continue with other tasks.

- name: Task1 Upgrading 10.04 to 12.04
  shell: do-release-upgrade -f DistUpgradeViewNonInteractive- name: Task2 restart machine after 10.04 to 12.04 upgrade
  shell: reboot
  async: 0
  poll: 0
- name: Task3 waiting for server to come back after 10.04 to 12.04
  local_action: wait_for host={{ ansible_ssh_host }} state=started

Task1 is slef explanatory which use shell module to upgrade the OS. Task two uses shell module to reboot the machine and task three use local_action which will be done one the host where Ansible is running this play book and wait_for will wait for something to happen. Here we are waiting for remote host to come on-line.

Other options you may include are

 port –On which port Ansible can check on remote machine once the “timeout” is completed

delay –Dont check it for number of seconds

timeout –Wait for something to happen on remote machine

connect_timeout — maximum number of seconds to wait for a connection to happen before closing and retrying

The third task can be written as below.

 local_action: wait_for host={{ ansible_ssh_host }}                         state=started                         port=22                         delay=30                         timeout=300                         connect_timeout=15

Wait on Ansible host for 300 seconds to check if host started or not. Once host is started wait for 30 seconds to check for 22 port is open or not and timeout that ssh connection after 15 seconds.

Some may use inventory_hostname instead of ansible_ssh_host variable to poll host identity. If you want to check on both variables you can use or operator and make one variable a default one as shown below.

Read Full Post: http://www.linuxnix.com/ansible-reboot-server-in-play-book-and-wait-for-it-come-back/