Automating Stop-Start Procedures with Ansible

Projects: c2platform/ansible, c2platform.core, c2platform.wincore

This how-to guide explains how to automate the stop, start, and restart procedures for the “WinApp” system within the C2 reference implementation and Ansible inventory project: c2platform/ansible. The inventory project leverages a versatile restart role from the c2platform.core Ansible collection. This role allows us to implement stop-start routines by configuring a list called restart_config.

In the inventory project this list is used to define five distinct routines: stop, start, restart, maintenance-start, and maintenance-stop. You can find the configuration in group_vars/winapp_system/restart_config.yml within the c2platform/ansible project. The crucial point here is that using a flexible and generic role like restart enables the implementation of stop-start routines without the need to create or manage separate Ansible roles or collections for each application. A single well-designed role can be used across hundreds or even thousands of applications, making it suitable for larger organizations with diverse and non-standardized application portfolios.

Overview

WinApp System

The WinApp System consists of four nodes:

1. Three LXD containers: c2d-rproxy1, c2d-tomcat1, c2d-db1. These containers run a reverse proxy based on Apache, an application server based on Tomcat, and a PostgreSQL database, respectively.
2. One MS Windows VirtualBox VM running IIS.

   

Managing WinApp

The diagram below illustrates the setup for managing the WinApp application. Using the Automation Controller from Ansible Automation Platform (AAP), an engineer can stop and start the WinApp application.

When the engineer runs the “stop” play, the Automation Controller (AWX):

1. Performs a Source Update, downloading the latest version of the content from the Ansible inventory project c2platform/ansible hosted on GitLab.com 
2. Downloads Ansible roles and collections from Galaxy.
3. Performs an Inventory update based on hosts-dev.ini, updating information about servers and hosts while reading configurations from group_vars, including the restart configuration in group_vars/winapp_system/restart.yml.
4. Executes one of the available Ansible plays to manage the system, which includes c2d-rproxy1, c2d-tomcat1, c2d-winapp1, and c2d-db1. Five plays correspond to each routine defined in group_vars/winapp_system/restart.yml.

Routine	Job Template	Play	Description
Stop	c2d-winapp-stop	stop.yml	Displays a maintenance message and stops WinApp services (IIS, Tomcat, and PostgreSQL).
Start	c2d-winapp-start	start.yml	Removes the maintenance message and starts WinApp services.
Restart	c2d-winapp-restart	restart.yml	Combines tasks of stop and start routine.
Maintenance Start	c2d-winapp-maintenance-start	maintenance-start.yml	Displays a maintenance message, blocking end-user access. No WinApp services are stopped.
Maintenance Stop	c2d-winapp-maintenance-stop	maintenance-stop.yml	Removes the maintenance message and makes the application available again for end users.

Prerequisites

Create the reverse and forward proxy c2d-rproxy1.

c2
unset PLAY  # ensure all plays run
vagrant up c2d-rproxy1

For more information about the various roles that c2d-rproxy1 performs in this project:

• Setup Reverse Proxy and CA server
• Setup SOCKS proxy
• Managing Server Certificates as a Certificate Authority
• Setup DNS for Kubernetes

Setting Up WinApp

Run the following commands to create and start the necessary nodes of the WinApp System:

c2
export BOX="c2d-winapp1 c2d-db1 c2d-tomcat1"
vagrant up $BOX

Setting Up AWX

To create the Automation Hub (AWX) node c2d-awx1, execute the following command:

c2
vagrant up c2d-awx1

This command will create a Kubernetes cluster and deploy AWX on it using the AWX operator.

This command creates a Kubernetes cluster and deploys AWX on it using the AWX operator. For detailed instructions on creating this node, refer to Setup the Automation Controller ( AWX ) using Ansible

Verifying WinApp and AWX

With c2d-rproxy1 node running, you should be able to access the following links:

1. https://winapp-iis.c2platform.org/is-alive  : This link should display “Apache is alive,” indicating that the reverse proxy c2d-rproxy1 is operational.
2. https://winapp-tomcat.c2platform.org/  : Accessing this link will display the Tomcat default page, confirming that c2d-tomcat1 is running.
3. https://winapp-iis.c2platform.org/  : Accessing this link should show the default IIS home page which verifies that IIS is running on c2d-winapp1

To verify the AWX installation:

1. Access the AWX web UI at https://awx.c2platform.org/  . Log in with the default credentials: username admin and password secret.
2. Confirm that the AWX dashboard loads without errors.

Executing Stop-Start Routines

To execute the routines using AWX:

1. Access the AWX web UI at https://awx.c2platform.org/  and log in as admin using password secret.
2. Navigate to Templates and select the appropriate Job Template based on the routine you want to execute. For example, select c2d-winapp-stop for the “Stop” routine.
3. Click Launch to execute the routine. You can monitor the job’s progress in AWX.

Stop WinApp

To stop WinApp launch the c2d-winapp-stop job. After it completes go to https://winapp-tomcat.c2platform.org/  and https://winapp-iis.c2platform.org/  there now is a message

We’ll be back soon!

To see the AWX job output click

==> c2d-rproxy1: Running provisioner: shell...
    c2d-rproxy1: Running: inline script
==> c2d-rproxy1: Running provisioner: ansible...
    c2d-rproxy1: Running ansible-playbook...

PLAY [Stop] ********************************************************************

TASK [Gathering Facts] *********************************************************
ok: [c2d-rproxy1]

TASK [Gather facts → restart_facts_hosts] **************************************

TASK [c2platform.core.facts : Set facts] ***************************************
ok: [c2d-rproxy1]

TASK [c2platform.core.facts : Gather facts] ************************************
ok: [c2d-rproxy1] => (item=c2d-rproxy1)
ok: [c2d-rproxy1 -> c2d-db1(1.1.4.207)] => (item=c2d-db1)
ok: [c2d-rproxy1 -> c2d-winapp1(1.1.8.146)] => (item=c2d-winapp1)
ok: [c2d-rproxy1 -> c2d-tomcat1(1.1.4.151)] => (item=c2d-tomcat1)

TASK [c2platform.core.facts : Set facts] ***************************************
skipping: [c2d-rproxy1]

TASK [c2platform.core.facts : Set facts] ***************************************
ok: [c2d-rproxy1]

TASK [c2platform.core.restart : Set fact restart_hosts_unreachable] ************
ok: [c2d-rproxy1]

TASK [c2platform.core.restart : Show unreachable hosts] ************************
skipping: [c2d-rproxy1]

TASK [c2platform.core.restart : Fail when unreachable] *************************
skipping: [c2d-rproxy1]

TASK [c2platform.core.restart : Set fact restart_hosts_reachable] **************
ok: [c2d-rproxy1]

TASK [c2platform.core.restart : include_tasks] *********************************
skipping: [c2d-rproxy1] => (item=Verify Tomcat maintenance message → c2d-rproxy1)
skipping: [c2d-rproxy1] => (item=Verify IIS maintenance message → c2d-rproxy1)
skipping: [c2d-rproxy1] => (item=Start PostgreSQL → c2d-db1)
skipping: [c2d-rproxy1] => (item=Wait PostgreSQL → c2d-db1)
skipping: [c2d-rproxy1] => (item=Start Tomcat → c2d-tomcat1)
skipping: [c2d-rproxy1] => (item=Start IIS → c2d-winapp1)
skipping: [c2d-rproxy1] => (item=Verify IIS port started → c2d-winapp1)
skipping: [c2d-rproxy1] => (item=Wait IIS → c2d-winapp1)
skipping: [c2d-rproxy1] => (item=Stop maintenance → c2d-rproxy1)
skipping: [c2d-rproxy1] => (item=Reload reverse proxy → c2d-rproxy1)
skipping: [c2d-rproxy1] => (item=Verify Tomcat online → c2d-rproxy1)
skipping: [c2d-rproxy1] => (item=Verify IIS maintenance message → c2d-rproxy1)
included: /home/ostraaten/git/gitlab/c2/ansible-dev-collections/ansible_collections/c2platform/core/roles/restart/tasks/copy.yml for c2d-rproxy1 => (item=Start maintenance → c2d-rproxy1)
included: /home/ostraaten/git/gitlab/c2/ansible-dev-collections/ansible_collections/c2platform/core/roles/restart/tasks/service.yml for c2d-rproxy1 => (item=Reload reverse proxy → c2d-rproxy1)
included: /home/ostraaten/git/gitlab/c2/ansible-dev-collections/ansible_collections/c2platform/core/roles/restart/tasks/win_service.yml for c2d-rproxy1 => (item=Stop IIS → c2d-winapp1)
included: /home/ostraaten/git/gitlab/c2/ansible-dev-collections/ansible_collections/c2platform/core/roles/restart/tasks/win_wait_for.yml for c2d-rproxy1 => (item=Verify IIS port drained → c2d-winapp1)
included: /home/ostraaten/git/gitlab/c2/ansible-dev-collections/ansible_collections/c2platform/core/roles/restart/tasks/service.yml for c2d-rproxy1 => (item=Stop Tomcat → c2d-tomcat1)
included: /home/ostraaten/git/gitlab/c2/ansible-dev-collections/ansible_collections/c2platform/core/roles/restart/tasks/service.yml for c2d-rproxy1 => (item=Stop PostgreSQL → c2d-db1)

TASK [c2platform.core.restart : Start maintenance] *****************************
changed: [c2d-rproxy1]

TASK [c2platform.core.restart : Reload reverse proxy] **************************
changed: [c2d-rproxy1]

TASK [c2platform.core.restart : Stop IIS] **************************************
changed: [c2d-rproxy1 -> c2d-winapp1(1.1.8.146)]

TASK [c2platform.core.restart : Verify IIS port drained] ***********************
ok: [c2d-rproxy1 -> c2d-winapp1(1.1.8.146)]

TASK [c2platform.core.restart : Stop Tomcat] ***********************************
changed: [c2d-rproxy1 -> c2d-tomcat1(1.1.4.151)]

TASK [c2platform.core.restart : Stop PostgreSQL] *******************************
changed: [c2d-rproxy1 -> c2d-db1(1.1.4.207)]

PLAY RECAP *********************************************************************
c2d-rproxy1                : ok=18   changed=5    unreachable=0    failed=0    skipped=3    rescued=0    ignored=0

When an engineer executes the “stop” routine for the WinApp application, the following steps are taken:

1. Start maintenance: This task is executed on the c2d-rproxy1 server. It involves changing the Apache2 reverse proxy configuration to display a maintenance message to end users attempting to access the application. The maintenance message reads: “We’ll be back soon.”
2. Reload reverse proxy: Also executed on c2d-rproxy1, this task reloads the Apache2 service to activate the maintenance message. This action effectively blocks end users from accessing the application during maintenance.
3. Stop IIS: This task is executed on c2d-winapp1, which is the server hosting the IIS (Internet Information Services) service for the WinApp application. The IIS service is stopped to prevent it from handling incoming requests.
4. Verify IIS port drained: Also on c2d-winapp1, Ansible monitors the IIS port, waiting for it to drain. This step ensures that no active connections are left on the IIS service before proceeding with further maintenance tasks.
5. Stop Tomcat: On c2d-tomcat1, which is the server hosting the Tomcat service for the WinApp application, the Tomcat service is stopped. This action halts the processing of Java-based components of the application.
6. Stop PostgreSQL: On c2d-db1, which is the database server for the WinApp application, the PostgreSQL service is stopped. This action prevents any database transactions or updates while maintenance is ongoing.

   

Start, Restart etc.

The functionality and operation of the start, restart, maintenance-start, and maintenance-stop routines are straightforward and intuitive, as defined in the restart_config. Consequently, we won’t delve into the detailed steps for each of these routines individually.

Feel free to explore and experiment with these routines based on your specific requirements and configurations. You can easily initiate and test these routines to observe their behavior and verify their functionality. This hands-on approach will provide you with a practical understanding of how they work within your environment.

By leveraging the flexible restart role and the configuration options available in restart_config, you have the flexibility to adapt and fine-tune these routines to suit your organization’s unique needs. This versatility makes it convenient to manage a wide range of applications efficiently.

Managing with Vagrant

You also have the option to execute these plays directly from the command line using Vagrant. This approach offers a convenient way to manage WinApp.

c2
PLAY=plays/mgmt/restart/stop.yml \
vagrant provision c2d-rproxy1 | tee provision.log

Manage WinApp using Ansible CLI

If you prefer, you can choose to execute these routines using the Ansible CLI without relying on Vagrant.

c2
ansible-playbook plays/mgmt/restart/restart.yml -i hosts-dev.ini

Please ensure that you have the correct SSH configuration in your ~/.ssh/config file, which is essential for smooth execution. Here’s an example configuration:

Host c2d-*
  ProxyCommand ssh 1.1.4.205 -W %h:%p
  User vagrant
  IdentityFile ~/.vagrant.d/insecure_private_key
  StrictHostkeyChecking no
  UserKnownHostsFile /dev/null
  LogLevel INFO
  Compression yes
  ServerAliveInterval 10
  ServerAliveCountMax 10

For further details, you can refer to the documentation on Using Ansible without Vagrant.

Review

In the C2 Inventory project c2platform/ansible.

1. The file group_vars/winapp_system/restart_config.yml serves as the configuration hub for defining the five “stop-start” routines using the restart_config list. These routines are implemented through various Ansible modules, including copy, service, uri, wait_for, win_powershell, win_service and win_wait_for.
2. In the plays/mgmt/restart folder, a dedicated play has been created for each routine. These plays exclusively incorporate the restart role, an integral component of the Ansible collection c2platform.core.

Within the C2 Ansible Collection project c2platform.core:

1. The restart Ansible role plays a central role in orchestrating these routines. This role leverages a range of Ansible modules to carry out the necessary tasks efficiently and effectively.

These well-structured configurations and roles streamline the management of the WinApp system, enhancing automation and control.