Managing Secrets with Ansible Vault in AAP / AWX

Projects: c2platform/ansible , c2platform.core

Problem

Ansible Automation Platform lacks built-in support for Ansible Vault, which creates challenges when integrating Vault-encrypted files into the group_vars directory. This limitation affects inventory projects that rely on Git as the foundation for AAP / AWX deployments, often leading to update failures because you cannot configure an Ansible Vault secret for such projects.

AAP / AWX avoids built-in Vault support to prevent exposing encrypted secrets to platform users or interfaces. This exposure could compromise sensitive data confidentiality, as users might gain unintended access to decrypted secrets during job execution or template management. As a result, secrets must be handled externally or through custom workarounds to maintain security.

Context

Managing secrets is essential in Ansible projects, and Ansible Vault provides a standard, built-in solution. However, using it effectively in AAP / AWX requires specific setups and considerations to ensure compatibility.

In Ansible and automation projects—especially in environments like the Dutch government—a dedicated secrets management tool may not always be available. In these cases, Ansible Vault is a simple, practical choice. It integrates seamlessly with Ansible (no extra costs or licenses) and is easy to replace if a more advanced solution becomes available later.

Solution

To manage secrets securely in AAP / AWX without built-in Vault support, use a custom directory for Vault-encrypted files and a dedicated Ansible role to load them dynamically. This approach avoids storing secrets in group_vars, which can cause sync failures in inventory projects due to the inability to provide a Vault password.

Follow these guidelines:

1. Create a Custom Secrets Directory: Avoid the standard group_vars folder, as it doesn’t work reliably with AAP / AWX (you can’t supply a Vault password to inventory projects). Instead, create a folder named secret_vars to store Vault-encrypted files. This naming convention is intuitive and helps teams understand its purpose.

2. Develop an Ansible “Secrets” Role: Build a generic role (e.g., c2platform.core.secrets) that loads secrets from secret_vars using tasks like include_vars. Ensure the role is flexible for both Ansible CLI and AAP / AWX environments. Configure it with a list like secrets_dirs to support multiple locations.

3. Use Vault in Development: Start using Ansible Vault even in development environments. For open-source setups (e.g., with simple passwords like secret), include required passwords in the vault early. This identifies necessary secrets for other environments without compromising security.

4. Establish a Procedure for Updating the Vault: To prevent merge conflicts (since Git can’t merge encrypted files), enforce a team-wide branching process:

   • Create a temporary branch (e.g., vault) from the latest master.
   • Update secrets in secret_vars on this branch.
   • Merge back to master immediately via a merge commit.
   • Sync to environment-specific branches (e.g., test) as needed.

5. Handle Merge Conflicts Locally: If conflicts occur (e.g., from process violations), resolve them manually:

   • Clone the conflicting branch locally.
   • Decrypt and compare vault contents from your branch and master using a diff tool.
   • Create a resolved version, encrypt it back into the vault.
   • Pull latest changes, commit, and push to resolve the conflict.

This method ensures security, prevents accidental exposure, and maintains compatibility across environments. Only authorized users should handle vault operations, with passwords managed securely.

Benefits

• Enhances security by isolating secrets and avoiding exposure in AAP / AWX interfaces.
• Improves project maintainability by preventing merge conflicts through structured procedures.
• Provides flexibility for both CLI and platform-based workflows without relying on built-in Vault support.
• Simplifies onboarding, as the secret_vars convention is clear and easy to adopt.

Alternatives (Optional)

While external tools like HashiCorp Vault or AWS Secrets Manager could be used for more advanced secrets management, Ansible Vault is preferred for its simplicity and zero-cost integration in environments without dedicated tools. It serves as a stepping stone that’s easy to migrate from later.

Examples and Implementation

Refer to the Ansible “Secrets” role c2platform.core.secrets  within the c2platform.core collection for an implementation example. This vault is included via include_vars in this role:

 roles/secrets/tasks/main.yml

20    - name: Include secrets
21      ansible.builtin.include_vars:
22        dir: "{{ secrets_dir_item['secrets_dir_item'] }}"
23      loop: >-
24        {{ secrets_dirs_stats['results']
25        | selectattr('stat.exists', 'equalto', True) }}        
26      loop_control:
27        label: "{{ secrets_dir_item['secrets_dir_item'] }}"
28        loop_var: secrets_dir_item
29      when: secrets_dir_item.stat.exists

The role utilizes the secrets_dirs list, which can be configured with multiple locations for the secret_vars folder. The following example works for both the Ansible CLI and AAP / AWX. When using AAP, AAP will place the vault in the specific location for example /runner/project/secret_vars/development. When testing Ansible playbooks on a Ansible development desktop, the vault will be found by the secrets role secret_vars/development.

 group_vars/all/secrets.yml

1---
2secrets_dirs:
3  - "{{ inventory_dir }}/secret_vars/{{ px_env }}"
4  - "/runner/project/secret_vars/{{ px_env }}"  #  awx / aap

Additional Information

• Ansible Vault: Securely manage secrets using Ansible Vault.
• Securely Accessing Ansible Vault in Development: Guideline for securely accessing Ansible Vault during development without storing passwords in plain files, using environment variables and scripts.
• Protecting sensitive data with Ansible vault — Ansible Community Documentation