Using GitLab Runner as Ansible Control Node

Problem

Many organizations aim to use AAP for orchestration tasks, but it may not always be available. In such cases, teams need an alternative control node to run Ansible playbooks for provisioning, while maintaining security, scalability, and integration with existing tools like GitLab. Choosing GitLab Runner for this purpose introduces challenges such as managing multiple pipelines, performing environment checks, and limiting provisioning to nodes that match the branch’s environment. These challenges do not exist with AAP, as it is specifically designed to configure and handle orchestration securely, including environment-specific controls, role-based access, and centralized management without custom pipeline logic.

Context

Ansible requires a control node to execute playbooks against target hosts. While AAP provides a robust, enterprise-grade solution, GitLab Runners can serve as a temporary or lightweight alternative. This is especially useful in environments with GitLab CI/CD, where runners can be configured with Ansible execution environments. The approach assumes a GitLab Runner with Ansible is already set up and focuses on creating pipelines for provisioning, using features like CI/CD inputs for flexibility¹.

This setup occurs in the context of an inventory project using group-based environments. GitLab Runner’s selection as the control node creates challenges like handling multiple pipelines for different tasks, validating environments to prevent unauthorized provisioning, and ensuring playbooks target only nodes in the matching environment group based on the branch.

Solution

Configure a GitLab CI/CD pipeline to use a runner as the Ansible control node. This involves setting up a main .gitlab-ci.yml file that includes conditional sub-files for default tasks or Ansible provisioning. Use CI/CD variables and inputs to make the pipeline dynamic and secure.

Modifying the Pipeline for Dual Workflows

Change the ordinary GitLab CI/CD pipeline for the inventory project to support two distinct workflows: one for standard tasks (e.g., linting, packaging) and an additional one specifically for Ansible provisioning. Define CI/CD inputs in the main .gitlab-ci.yml for selecting playbooks and CLI options. Create sub-files like .gitlab-ci/ansible.yml for provisioning and .gitlab-ci/default.yml for non-provisioning tasks. Use conditionals to include the appropriate sub-file based on pipeline triggers or variables.

Implementing Branch Validation for Provisioning

Add checks to ensure Ansible provisioning can only run on specific branches, namely those corresponding to group-based environments in the inventory project. The pipeline should fail if a user attempts to provision from the master branch or a feature branch. Set required project variables in GitLab settings, such as PX_ANSIBLE_ENVIRONMENTS for allowed branches. In the provisioning sub-file, include validation logic to check the current branch against this list before proceeding.

Limiting Playbook Execution to the Environment

When the Ansible pipeline runs from a correct environment branch, modify the ansible-playbook command to target only nodes in that environment group. Use the --limit option to restrict execution to hosts in the matching group, ensuring provisioning affects only the intended environment. Apply this limit automatically based on the branch name, promoting security and preventing accidental changes to unrelated hosts.

Follow these guidelines:

1. Define CI/CD inputs in the main .gitlab-ci.yml for selecting playbooks and CLI options.
2. Create a sub-file (e.g., .gitlab-ci/ansible.yml) for the provisioning stage, including variable checks, branch validation, and playbook execution with environment limits.
3. Create another sub-file (e.g., .gitlab-ci/default.yml) for non-provisioning tasks like linting and packaging, running only on merge requests or the default branch.
4. Set required project variables in GitLab settings, such as PX_ANSIBLE_ENVIRONMENTS for allowed branches.
5. Ensure the runner uses a secure image with Ansible installed, and apply access controls for security.

Benefits

• Enables Ansible provisioning without AAP, using existing GitLab infrastructure.
• Promotes flexibility with dynamic inputs for playbooks and options.
• Enhances security through branch validation and automatic limiting to environments.
• Supports scalability by integrating with GitLab’s CI/CD features for automation.

Alternatives

Directly running Ansible from a local machine or dedicated server is possible, but using GitLab Runners is preferred for integration with version control, automated pipelines, and scalability in team environments. If AAP becomes available, migrate for advanced features like job scheduling and RBAC.

Examples and Implementation

This section provides YAML examples for the pipeline files. Assume a GitLab project with Premium or Ultimate features for CI/CD inputs. The setup creates two pipelines: a default one for tasks like linting, and a provisioning one for Ansible playbooks.

Main Pipeline File

This main GitLab CI/CD pipeline file .gitlab-ci.yml defines inputs and includes other YAML files conditionally.

 .gitlab-ci.yml

---
spec:
  inputs:
    play:
      type: string
      description: Ansible playbook file
      options: ["plays/core/linux.yml", "plays/core/windows.yml", "none"]
      default: 'none'
    cli_options:
      type: string
      description: Ansible CLI options
      default: --check
---
variables:
  ANSIBLE_PLAY: "\"$[[ inputs.play ]]\""
  ANSIBLE_CLI_OPTIONS: "\"$[[ inputs.cli_options ]]\""

include:
  - local: .gitlab-ci/default.yml
    rules:
      - if: "\"$[[ inputs.play ]]\" == \"none\""
  - local: .gitlab-ci/ansible.yml
    rules:
      - if: "\"$[[ inputs.play ]]\" != \"none\""

Ansible Provisioning Pipeline

Handles provisioning with checks and playbook execution.

 .gitlab-ci/ansible.yml

---
default:
  image: registry.gitlab.com/c2platform/rws/ansible-execution-environment:0.1.25

stages:
  - validate
  - provision

validate:
  stage: validate
  script:
    - bash ./.gitlab-ci/scripts/validate.sh
  artifacts:
    paths:
      - variables.env
    expire_in: 1 hour

provision:
  stage: provision
  needs: [validate]
  script: |
    # Load exported variables
    source variables.env

    # Log and run the playbook
    echo "Running Ansible playbook: $CLEAN_PLAY, with options: $CLEAN_OPTIONS"
    echo "$ANSIBLE_CLI $CLEAN_PLAY $CLEAN_OPTIONS"
    # eval "$ANSIBLE_CLI \"$CLEAN_PLAY\" $CLEAN_OPTIONS"    

The validate.sh script validates that the branch is correct—in other words, it is an allowed environment branch (configured with PX_ANSIBLE_ENVIRONMENTS). And it ensures that --limit is used to target the group that matches the environment.

 .gitlab-ci/scripts/validate.sh

#!/bin/bash

# Check required variables
if [ -z "$PX_ANSIBLE_ENVIRONMENTS" ]; then
  echo "Error: Required variable PX_ANSIBLE_ENVIRONMENTS is not defined. Set it in CI/CD settings (e.g., 'development,test,acceptance,production'). Failing pipeline."
  exit 1
fi
if [ -z "$ANSIBLE_PLAY" ]; then
  echo "Error: Required variable ANSIBLE_PLAY is not defined. Set it to your playbook file/path (e.g., 'plays/core/linux.yml'). Failing pipeline."
  exit 1
fi
if [ -z "$ANSIBLE_CLI" ]; then
  ANSIBLE_CLI="ansible-playbook"  # Default if not set
  echo "ANSIBLE_CLI not defined. Defaulting to '$ANSIBLE_CLI'."
fi

# Strip surrounding double quotes from variables
clean_play="${ANSIBLE_PLAY#\"}"
clean_play="${clean_play%\"}"
clean_options="${ANSIBLE_CLI_OPTIONS#\"}"
clean_options="${clean_options%\"}"
clean_options=$(echo "$clean_options" | sed 's/""//g' | xargs)  # Trim extra spaces

# Validate branch
environments=$(echo "$PX_ANSIBLE_ENVIRONMENTS" | tr ',' ' ')
found=""
for env in $environments; do
  if [ "$CI_COMMIT_REF_NAME" = "$env" ]; then
    found="1"
    break
  fi
done
if [ -z "$found" ]; then
  echo "Error: Branch '$CI_COMMIT_REF_NAME' is not in allowed environments ($PX_ANSIBLE_ENVIRONMENTS). Failing pipeline."
  exit 1
fi

# Modify clean_options based on --limit
env="$CI_COMMIT_REF_NAME"
if echo "$clean_options" | grep -q -- --limit; then
  clean_options=$(echo "$clean_options" | sed -E "s/--limit ([^ ]+)/--limit \1:\&$env/")
else
  clean_options="$clean_options --limit $env"
fi

# Export to env file for next job
echo "CLEAN_PLAY=$clean_play" > variables.env
echo "CLEAN_OPTIONS=\"$clean_options\"" >> variables.env
echo "ANSIBLE_CLI=$ANSIBLE_CLI" >> variables.env
cat variables.env

Default Pipeline

For linting, packaging, and releasing. This is in .gitlab-ci/default.yml.

 .gitlab-ci/default.yml

---
default:
  image: registry.gitlab.com/c2platform/rws/ansible-execution-environment:0.1.25

variables:
  DOWNLOAD_SCRIPT: download.py
  COLLECTIONS_DIR: ansible-collections-tarball
  C2_PACKAGE_NAME: phx-ansible-collections

before_script:
  - python3 --version
  - pip3 --version
  - ansible --version
  - ansible-lint --version
  - yamllint --version

workflow:  # run the pipeline only on MRs and default branch
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

stages:
  - prepare
  - linters
  - package
  - release

yamllint:
  stage: linters
  script:
    - yamllint -c .yamllint .

ansible-lint:
  stage: linters
  script:
    - ansible-lint -c .ansible-lint

prepare:
  stage: prepare
  script:
    - C2_VERSION=$(grep -oP '\d+\.\d+\.\d+' CHANGELOG.md | head -1)
    - echo "C2_VERSION=$C2_VERSION" >> variables.env
  artifacts:
    reports:
      dotenv: variables.env

package:
  stage: package
  when: manual
  only:
    - master
  script:
    - |
      ansible-galaxy collection download -r collections/requirements.yml -p ./$COLLECTIONS_DIR
      cd $COLLECTIONS_DIR
      tar -czvf ../${C2_PACKAGE_NAME}.${C2_VERSION}.tar.gz .      
  artifacts:
    paths:
      - ${C2_PACKAGE_NAME}.${C2_VERSION}.tar.gz

publish:
  stage: package
  only:
    - master
  needs:
    - job: package
    - job: prepare
  script:
    - |
      env | grep C2
      curl --header "JOB-TOKEN: $CI_JOB_TOKEN" --upload-file ${C2_PACKAGE_NAME}.${C2_VERSION}.tar.gz \
        ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/${C2_PACKAGE_NAME}/${C2_VERSION}/${C2_PACKAGE_NAME}.${C2_VERSION}.tar.gz      

release:
  stage: release
  only:
    - master
  image: registry.gitlab.com/gitlab-org/release-cli:latest
  needs:
    - job: publish
    - job: package
  before_script: []
  script:
    - echo "Create release for $C2_VERSION"
  release:
    name: phx-inventory-$C2_VERSION
    description: PHX Ansible Inventory Project
    tag_name: $C2_VERSION
    ref: $CI_COMMIT_SHA
    assets:
      links:
        - name: "${C2_PACKAGE_NAME}.${C2_VERSION}.tar.gz"
          url: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/${C2_PACKAGE_NAME}/${C2_VERSION}/${C2_PACKAGE_NAME}.${C2_VERSION}.tar.gz"

Usage Notes

• Set CI/CD variables like PX_ANSIBLE_ENVIRONMENTS in project settings.
• The pipeline validates branches and limits playbooks to the current environment.
• Use a secure runner image and access controls for best practices.
• This setup scales well with GitLab without needing AAP.

Additional Information

For additional insights and guidance:

• Group-based Environments: Use a group-based approach to organize your Ansible inventory and variables for different environments.
• Ansible Inventory Project: A structured collection of files used for managing hosts and configurations. It typically includes inventory files, playbooks, host configurations, group variables, and Ansible vault files.