Docker-in-docker ( dind )

Run DinD locally for example to develop a GitLab CI/CD pipeline that will use this technique
Categories:
Tags:

Projects: c2platform/ansible

If you want to use DinD for example to build an execution environment it can be difficult to develop the pipeline using trial and error. Commit and push and see the results.

It make sense to develop, test the code locally.

So this how-to shows how you can similutate locally what happens in de pipeline by starting a container dind-service that offers a Docker service and that allows you to use Docker inside a cotnainer.s

You can use c2d-xtop to experiment with DinD. To create this run command below. See the play plays/dev/xtop.yml and configuration group_vars/xtop/main.yml.

vagrant up c2d-xtop

The play will start two containers, one named dind-service, one named docker.

vagrant@c2d-xtop:~$ docker ps
CONTAINER ID   IMAGE                                                   COMMAND                  CREATED          STATUS          PORTS           NAMES
5a8783ee37f4   docker:20.10.16                                         "docker-entrypoint.s…"   27 minutes ago   Up 27 minutes                   docker
8316dd08990c   registry.gitlab.com/c2platform/docker/c2d/dind:latest   "dockerd-entrypoint.…"   38 minutes ago   Up 38 minutes   2375-2376/tcp   dind-service
vagrant@c2d-xtop:~$

Inside the docker container you can run Docker commands.

vagrant@c2d-xtop:~$ docker exec -it docker sh
/ # docker pull ubuntu
Using default tag: latest
latest: Pulling from library/ubuntu
2ab09b027e7f: Pull complete
Digest: sha256:67211c14fa74f070d27cc59d69a7fa9aeff8e28ea118ef3babc295a0428a6d21
Status: Downloaded newer image for ubuntu:latest
docker.io/library/ubuntu:latest
/ #

If you created the c2d-gitlab node, you can also use that as sh

/ # docker pull registry.c2platform.org/c2platform/gitlab-docker-build:latest
latest: Pulling from c2platform/gitlab-docker-build
Digest: sha256:9a1289a3ae53088de59893efc9b998de1d6404bb99c7218119ea8d6bc22d9367
Status: Image is up to date for registry.c2platform.org/c2platform/gitlab-docker-build:latest
registry.c2platform.org/c2platform/gitlab-docker-build:latest
/ #

Setup Project Directories and Install Ansible

Configure Vagrant Sync Folder

Vagrant Sync Folders

---
- src: ../c2/docker
  target: /home/vagrant/images

export IMAGE_NAME="gitlab-runner"
export IMAGE_VERSION="0.0.1"
export IMAGE="$IMAGE_NAME:$IMAGE_VERSION"
docker build -t $IMAGE . | tee build.log
docker run -it --rm $IMAGE /bin/bash

(c2) vagrant@c2d-xtop:~/images/gitlab-runner$ docker run -it --rm $IMAGE
root@2f3b45e29a2e:/# ansible --version
ansible [core 2.15.0]
  config file = None
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /root/.virtualenv/c2d/lib/python3.10/site-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /root/.virtualenv/c2d/bin/ansible
  python version = 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] (/root/.virtualenv/c2d/bin/python)
  jinja version = 3.1.3
  libyaml = True
root@2f3b45e29a2e:/# ansible-builder --version
3.0.0
root@2f3b45e29a2e:/# git --version
git version 2.34.1

TEst met RWS

vagrant provision c2d-xtop
vagrant ssh c2d-xtop
docker ps
source ~/.virtualenv/c2d/bin/activate
git clone https://gitlab.com/c2platform/rws/ansible-execution-environment.git
cd ansible-execution-environment
rm -rf context
ansible-builder create
ansible-builder build

https://docs.docker.com/go/buildx/

---
collections:
  - name: ansible.windows
    version: 1.14.0
  - name: community.windows
    version: 2.0.0
  - name: awx.awx
    version: 22.4.0
  - name: checkmk.general
    version: 3.0.0
  - name: c2platform.core
    version: 1.0.8
  # - name: https://gitlab.com/c2platform/ansible-collection-core.git
  #  type: git
  #  version: master
  - name: c2platform.mgmt
    version: 1.0.2
  #- name: https://gitlab.com/c2platform/ansible-collection-mgmt.git
  #  type: git
  #  version: master
  - name: c2platform.gis
    version: 1.0.4
  #- name: https://gitlab.com/c2platform/rws/ansible-collection-gis.git
  #  type: git
 #   version: master
  - name: c2platform.wincore
    version: 1.0.5
  #- name: https://gitlab.com/c2platform/rws/ansible-collection-wincore.git
  #  type: git
  #  version: master
  - name: community.postgresql
    version: 2.4.3

(c2d) root@8ad3b66959cf:/ansible-execution-environment# rm -rf context (c2d) root@8ad3b66959cf:/ansible-execution-environment# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (c2d) root@8ad3b66959cf:/ansible-execution-environment# ansible-builder create Complete! The build context can be found at: /ansible-execution-environment/context (c2d) root@8ad3b66959cf:/ansible-execution-environment# ansible-builder build Running command: docker build -f context/Dockerfile -t ansible-execution-env:latest context Complete! The build context can be found at: /ansible-execution-environment/context (c2d) root@8ad3b66959cf:/ansible-execution-environment# docker images REPOSITORY TAG IMAGE ID CREATED SIZE ansible-execution-env latest c26229e6f4d7 2 minutes ago 632MB d772f9d37f78 3 minutes ago 638MB 3b2870c07300 5 minutes ago 286MB ghcr.io/ansible-community/community-ee-minimal 2.15.4-2 649292486077 5 months ago 270MB (c2d) root@8ad3b66959cf:/ansible-execution-environment#

Setup a Python Environment

Inside c2d-xtop, run the script /vagrant/scripts/penv.sh

source /vagrant/scripts/penv.sh

Install Ansible Builder and Navigator:

pip install ansible-builder
pip install ansible-navigator

Clone Project

cd /tmp
git clone git@gitlab.com:c2platform/rws/ansible-execution-environment.git
cd ansible-execution-environment

Build

rm -rf context
ansible-builder build --tag whatever_ee

Verify

ansible-navigator run test_localhost.yml \
  --execution-environment-image whatever_ee --mode stdout
Show me
(c2) vagrant@c2d-xtop:/ansible-execution-environment$ ansible-navigator run test_localhost.yml \
  --execution-environment-image whatever_ee --mode stdout
-----------------------------------------------------
Execution environment image and pull policy overview
-----------------------------------------------------
Execution environment image name:     whatever_ee:latest
Execution environment image tag:      latest
Execution environment pull arguments: None
Execution environment pull policy:    tag
Execution environment pull needed:    True
-----------------------------------------------------
Updating the execution environment
-----------------------------------------------------
Running the command: docker pull whatever_ee:latest
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match 'all'

PLAY [Gather and print local Ansible / Python facts] ***************************

TASK [Gathering Facts] *********************************************************
ok: [localhost]

TASK [Print Ansible version] ***************************************************
ok: [localhost] => {
    "ansible_version": {
        "full": "2.15.0",
        "major": 2,
        "minor": 15,
        "revision": 0,
        "string": "2.15.0"
    }
}

TASK [Print Ansible Python] ****************************************************
ok: [localhost] => {
    "ansible_playbook_python": "/usr/bin/python3"
}

TASK [Print Ansible Python version] ********************************************
ok: [localhost] => {
    "ansible_python_version": "3.11.4"
}

PLAY RECAP *********************************************************************
localhost                  : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
What is Docker-in-docker ( DinD )?

Docker-in-Docker (DinD) is a common technique used in GitLab CI/CD pipelines on Kubernetes to run Docker commands within Docker containers.


Last modified November 14, 2024: guideline tags en fme flow tags RWS-353 (ed0ed3f)