GitLab Pipelines in Kubernetes

Running GitLab pipelines in Kubernetes using local GitLab instance c2d-gitlab.
Categories:
Tags:

This how-to describes how we can run GitLab CI/CD pipelines on Kubernetes using a self-managed / self-hosted GitLab CE installation. It basically moves the example project examples/kubernetes/gitlab-docker-build that uses gitlab.com  to a local GitLab CE instance running on c2d-gitlab.

Overview

Prerequisites

Create the reverse and forward proxy c2d-rproxy1.

c2
unset PLAY  # ensure all plays run
vagrant up c2d-rproxy1

For more information about the various roles that c2d-rproxy1 performs in this project:

For the first two prerequisites running command vagrant up c2d-rproxy1 c2d-gitlab should be enough.

Custom docker-in-docker (dind) images

The project examples/kubernetes/gitlab-docker-build that is imported in our local GitLab CE instance running on c2d-ks1 uses custom Docker images see .gitlab-ci.yml. The related projects that create these two images are:

The only customization done in these projects is to import the c2d root ca bundle c2.crt  see Dockerfile  .

Without this customization Docker commands will fail with message

x509: certificate signed by unknown authority

Import GitLab project

Create public namespace https://gitlab.c2platform.org/c2platform  and then Import project → Repository by URL

PropertyValue
URLhttps://gitlab.com/c2platform/examples/kubernetes/gitlab-docker-build.git 
Visibility LevelPublic

GitLab Runner

Get registration token

Navigate to https://gitlab.com/c2platform/examples/kubernetes/gitlab-docker-build  and Settings → CI/CD  and then Runners and copy the registration token.

Edit values.yml

Put the registration token in values.yml.

vagrant ssh c2d-ks1
nano /vagrant/doc/howto-kubernetes-gitlab-local/values.yml  # and update registration token

Install runner

Run the script gitlab-runner.sh

source /vagrant/doc/howto-kubernetes-gitlab-local/gitlab-runner.sh

Verify

Check the GitLab Runner pod log. It should show message

Runner registered successfully

Show me
vagrant@c2d-ks1:~/scripts/microk8s/gitlab/gitlab-runner$ kubectl logs -f gitlab-runner-c46457f8b-6dr7g
Registration attempt 1 of 30
Runtime platform                                    arch=amd64 os=linux pid=14 revision=456e3482 version=15.10.0
WARNING: Running in user-mode.
WARNING: The user-mode requires you to manually start builds processing:
WARNING: $ gitlab-runner run
WARNING: Use sudo for system-mode:
WARNING: $ sudo gitlab-runner...

WARNING: There might be a problem with your config
jsonschema: '/runners' does not validate with https://gitlab.com/gitlab-org/gitlab-runner/common/config#/$ref/properties/runners/type: expected array, but got null
Created missing unique system ID                    system_id=r_pjr46VlRtCXO
Merging configuration from template file "/configmaps/config.template.toml"
WARNING: Support for registration tokens and runner parameters in the 'register' command has been deprecated in GitLab Runner 15.6 and will be replaced with support for authentication tokens. For more information, see https://gitlab.com/gitlab-org/gitlab/-/issues/380872
Registering runner... succeeded                     runner=GR1348941TvUfy4ig
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!

Configuration (with the authentication token) was saved in "/home/gitlab-runner/.gitlab-runner/config.toml"
Runtime platform                                    arch=amd64 os=linux pid=7 revision=456e3482 version=15.10.0
Starting multi-runner from /home/gitlab-runner/.gitlab-runner/config.toml...  builds=0
WARNING: Running in user-mode.
WARNING: Use sudo for system-mode:
WARNING: $ sudo gitlab-runner...

WARNING: There might be a problem with your config
jsonschema: '/runners/0/kubernetes/node_tolerations' does not validate with https://gitlab.com/gitlab-org/gitlab-runner/common/config#/$ref/properties/runners/items/$ref/properties/kubernetes/$ref/properties/node_tolerations/type: expected object, but got null
Configuration loaded                                builds=0
listen_address not defined, metrics & debug endpoints disabled  builds=0
[session_server].listen_address not defined, session endpoints disabled  builds=0
Initializing executor providers                     builds=0

If you enter the pod take note of files /home/gitlab-runner/.gitlab-runner/certs/gitlab.c2platform.org.crt and /home/gitlab-runner/.gitlab-runner/config.toml

Show me
vagrant@c2d-ks1:~$ kubectl exec -it gitlab-runner-c46457f8b-6dr7g -- sh
/ $ ls /home/gitlab-runner/.gitlab-runner/certs
gitlab.c2platform.org.crt
/ $ cat /home/gitlab-runner/.gitlab-runner/config.toml
concurrent = 10
check_interval = 30
log_level = "info"
shutdown_timeout = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "gitlab-runner-c46457f8b-6dr7g"
  url = "https://gitlab.c2platform.org/"
  id = 1
  token = "yU1z6fGDjbr2pngVGh3A"
  token_obtained_at = 2023-03-28T05:57:34Z
  token_expires_at = 0001-01-01T00:00:00Z
  executor = "kubernetes"
  [runners.cache]
    MaxUploadedArchiveSize = 0
  [runners.kubernetes]
    host = ""
    bearer_token_overwrite_allowed = false
    image = "ubuntu:20.04"
    namespace = "gitlab-runner"
    namespace_overwrite_allowed = ""
    privileged = true
    node_selector_overwrite_allowed = ""
    pod_labels_overwrite_allowed = ""
    service_account_overwrite_allowed = ""
    pod_annotations_overwrite_allowed = ""
    [runners.kubernetes.pod_security_context]
    [runners.kubernetes.init_permissions_container_security_context]
    [runners.kubernetes.build_container_security_context]
    [runners.kubernetes.helper_container_security_context]
    [runners.kubernetes.service_container_security_context]
    [runners.kubernetes.volumes]

      [[runners.kubernetes.volumes.empty_dir]]
        name = "docker-certs"
        mount_path = "/certs/client"
        medium = "Memory"
    [runners.kubernetes.dns_config]
/ $

Of course you should also now the runner under Runners section via CI/CD settings  .

Run pipeline

Now that there a project runner is available we can start pipeline via CI/CD → Pipelines  .


Last modified June 5, 2024: fme verify https://gsd-fme-core.internal.c2platform.org/fmeserver/ RWS-859 (ce8abda)