---
title: "Common Approaches for Building Container Images on K8s: Docker (DinD/DooD) / Kaniko / BuildKit"
description: "Approaches and trade-offs for building container images on K8s — covering DinD, DooD, Kaniko, BuildKit, and worked examples."
canonical_url: "https://blog.markkulab.net/en/post/k8s-build-image-methods"
author: "Mark Ku"
author_url: "https://blog.markkulab.net/en/author/mark-ku"
site: "Mark Ku's Tech Notes"
date_published: "2025-09-02 02:01:00 +0800"
category: "DevOps"
tags: ["kubernetes", "docker", "ci/cd", "gitlab runner", "kaniko", "buildkit", "dind", "dood", "devops"]
language: "en"
license: "CC BY 4.0"
license_url: "https://creativecommons.org/licenses/by/4.0/"
attribution: "when reusing or quoting, credit the author and link back to the original"
---

# Common Approaches for Building Container Images on K8s: Docker (DinD/DooD) / Kaniko / BuildKit

## Overview

This article quickly compares **DinD**, **DooD**, **Kaniko**, **BuildKit**, and **Buildah** — and provides a minimal viable configuration for GitLab Runner on K8s with common gotchas resolved.

> **TL;DR:** For building images on K8s — if you want compatibility and security, prioritize **Kaniko**; if you want performance and caching, pick **BuildKit**; only use DinD/DooD in trusted environments or for one-off needs.

## Tool comparison

| Tool | Daemon required | Privileged required | Performance | Compatibility | Best for |
|------|---------------|-------------------|------|--------|----------|
| **DinD** | ✅ | ✅ | Medium | High | Small-team quick CI |
| **DooD** | ✅ (host) | ✅ | High | High | Internal-use CI |
| **Kaniko** | ❌ | ❌ | Medium | High | Cloud-native CI/CD |
| **BuildKit** | ✅ (buildkitd) | ✅/rootless | High | High | Need caching/performance |
| **Buildah** | ❌ | ❌ (rootless OK) | Medium | High | OpenShift / Red Hat ecosystems |

## DinD vs DooD

People often get confused by these two terms — the diagram below makes the difference clear quickly:

![](https://blog.markkulab.net/content/markku/posts/k8s-build-image-methods/images/attachments/372ac1ee-0f92-480b-a43c-188bfabdd491.png)

### DinD (Docker-in-Docker)

* In CI/CD pipelines, no need to mount docker.sock
* Can connect via TCP
* Suitable for isolated CI/CD environments

**Pros:**
- Simple setup, clear dependencies, fully compatible with Docker commands
- Container-internal isolation, convenient for one-off testing

**Cons:**
- Requires privileged mode — higher risk
- Medium performance — nested cgroups bring extra overhead
- Often runs into network and permission limits on managed K8s/VMs in the cloud

### DooD (Docker-outside-of-Docker)

* The container doesn't run its own Docker
* Directly mounts the host's Docker socket (`/var/run/docker.sock`) into the container
* Better performance, but lower security

**Pros:**
- Best performance — reuses host's Docker cache
- Fast startup

**Cons:**
- Equivalent to exposing host Docker permissions to the container — unsuitable for multi-tenancy
- Tied to the node — K8s portability and flexibility are reduced

#### Why do most online examples use Docker 20.10?

Docker 20.10 was released in late 2020 as a long-term-stable release with continuous security updates. Many Linux distros bundle this version, ecosystem support is strongest, and most automation tools (GitLab Runner, Drone, Jenkins, etc.) were originally tested against it — so it's the most stable and reliable.

Starting from Docker 23.x, many features have changed back and forth, with config and APIs shifting — leading to legacy example code breaking outright. On Kubernetes or GitLab Runner, you may hit build failures.

## Deployment guide for the various build approaches

### 1. K8s DinD deployment (GitLab Runner needs privileged=true)

I tried many times — DinD in K8s, I couldn't get it to work in the same Pod. The principle should be connecting through Docker remote management port 2375 to build, but given my company's security considerations, I didn't pursue it further.

### 2. K8s DooD deployment

Need to mount `/var/run/docker.sock` and share the host's Docker daemon.

#### Prerequisites

Install Docker on the host machine.

#### GitLab CI config

```yaml
stages:
  - build

variables:
  IMAGE: $CI_REGISTRY_IMAGE/$CI_BUILD_REF_NAME:$CI_PIPELINE_ID     
  K8S_NAMESPACE: "kong-api-gateway"
  KONG_SECRET_NAME: "kong-api-gateway-secret"
  DOCKER_TLS_CERTDIR: ""
  DOCKER_DRIVER: overlay2
 
build:
  stage: build
  image: docker:20.10
  services:
    - name: docker:20.10
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - echo "=== Building Kong API Gateway Docker image (DooD) ==="
    - echo "Target image: ${IMAGE}"
    - echo "Build context: ${CI_PROJECT_DIR}"
    - docker build -f Dockerfile.kong -t "${IMAGE}" "${CI_PROJECT_DIR}"
    - docker push "${IMAGE}"
  only:
    - main
  tags:
    - K8s-Runner
```

#### GitLab Runner (Helm Values)

```yaml
gitlabUrl: https://gitlab.com
runnerRegistrationToken: "xxx"
unregisterRunners: true

fullnameOverride: "k8s-cd-gitlab-runner"

serviceAccount:
  create: true
  name: gitlab-runner

runners:
  privileged: true
  tags: "deploy"
  config: |
    [[runners]]
      [runners.kubernetes]
        image = "docker:20.10"
        service_account = "gitlab-runner"
        service_account_overwrite_allowed = ".*"
        [runners.kubernetes.pod_security_context]
          run_as_non_root = false
          run_as_user = 0
        [runners.kubernetes.container_security_context]
          privileged = true
        [runners.kubernetes.resources]
          limits = { "cpu" = "1000m", "memory" = "2Gi" }
          requests = { "cpu" = "500m", "memory" = "1Gi" }
        [runners.kubernetes.environment]
          DOCKER_OPTS = "--insecure-registry 192.168.50.57:30000"
        [[runners.kubernetes.volumes.host_path]]
          name = "docker-socket"
          mount_path = "/var/run/docker.sock"
          host_path = "/var/run/docker.sock"
          mount_propagation = "HostToContainer"

securityContext:
  allowPrivilegeEscalation: true
  readOnlyRootFilesystem: false
  runAsNonRoot: false
  privileged: true
  capabilities:
    add: ["SYS_ADMIN"]

podSecurityContext:
  runAsUser: 0
  fsGroup: 0
```

#### RBAC config

```yaml
# gitlab-runner-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: gitlab-runner
  namespace: k8s-cd-gitlab-runner
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: k8s-cd-gitlab-runner
  name: gitlab-runner-role
rules:
- apiGroups: [""]
  resources: ["pods", "pods/attach", "pods/exec", "pods/log", "pods/portforward", "pods/proxy", "pods/status"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: [""]
  resources: ["secrets", "configmaps", "persistentvolumeclaims", "services", "endpoints"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: [""]
  resources: ["events"]
  verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: [""]
  resources: ["namespaces"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
  resources: ["deployments", "statefulsets", "daemonsets"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["batch"]
  resources: ["jobs", "cronjobs"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["extensions"]
  resources: ["deployments", "statefulsets", "daemonsets"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: gitlab-runner-rolebinding
  namespace: k8s-cd-gitlab-runner
subjects:
- kind: ServiceAccount
  name: gitlab-runner
  namespace: k8s-cd-gitlab-runner
roleRef:
  kind: Role
  name: gitlab-runner-role
  apiGroup: rbac.authorization.k8s.io
```

#### Deployment commands

```bash
# Deploy
helm repo add gitlab https://charts.gitlab.io
helm repo update
kubectl create namespace k8s-cd-gitlab-runner
kubectl apply -f ./rbac.yaml 
helm install gitlab-runner -f values.yaml gitlab/gitlab-runner --namespace k8s-cd-gitlab-runner --create-namespace

# Update
helm upgrade gitlab-runner -f values.yaml gitlab/gitlab-runner --namespace k8s-cd-gitlab-runner
```

> **Note:** This method works on self-hosted Ubuntu K8s, but on RKE2 some security settings may prevent binding `/var/run/docker.sock`.

### 3. Kaniko deployment

Best security: builds don't need a Docker daemon or privileged mode. Excellent compatibility on managed K8s, supports common Dockerfile commands, easy migration and debugging — but builds are 20–30% slower than DinD or DooD.

#### Helm Values config

```yaml
gitlabUrl: https://gitlab.com/
runnerRegistrationToken: ""  # The token you see in GitLab -> Settings -> CI/CD -> Runners
unregisterRunners: true

fullnameOverride: "k8s-cd-gitlab-runner"

serviceAccount:
  create: false
  name: gitlab-runner

runners:
  privileged: false
  tags: "K8s-Runner"
  config: |
    [[runners]]
      [runners.kubernetes]
        image = "gcr.io/kaniko-project/executor:debug"
        service_account = "gitlab-runner"
        service_account_overwrite_allowed = ".*"
        privileged = false
```

#### Deployment command

```bash
helm install gitlab-runner -f values.yaml gitlab/gitlab-runner --namespace k8s-cd-gitlab-runner --create-namespace
```

#### GitLab CI config

```yaml
stages:
  - build

variables:
  IMAGE: $CI_REGISTRY_IMAGE/$CI_BUILD_REF_NAME:$CI_PIPELINE_ID
  K8S_NAMESPACE: "kong-api-gateway"
  KONG_SECRET_NAME: "kong-api-gateway-secret"

build:
  stage: build
  image: gcr.io/kaniko-project/executor:debug
  variables:
    DOCKER_CONFIG: /kaniko/.docker
  before_script:
    - mkdir -p /kaniko/.docker
    - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
  script:
    - echo "=== Building Kong API Gateway Docker image (Kaniko) ==="
    - echo "Target image: ${IMAGE}"
    - echo "Build context: ${CI_PROJECT_DIR}"
    - /kaniko/executor --context "${CI_PROJECT_DIR}" --dockerfile "Dockerfile.kong" --destination "${IMAGE}" --cache=true --cleanup
  only:
    - main
  tags:
    - K8s-Runner

create-secret:
  stage: create-secret
  image:
    name: bitnami/kubectl:latest
  script:
    - echo "=== Preparing target namespace and image-pull secret ==="
    - kubectl get namespace ${K8S_NAMESPACE} || kubectl create namespace ${K8S_NAMESPACE}
    - kubectl delete secret ${KONG_SECRET_NAME} -n ${K8S_NAMESPACE} --ignore-not-found=true || true
    - kubectl create secret docker-registry ${KONG_SECRET_NAME} \
        --docker-server=$CI_REGISTRY \
        --docker-username=$CI_REGISTRY_USER \
        --docker-password=$CI_REGISTRY_PASSWORD \
        --docker-email=none \
        -n ${K8S_NAMESPACE}
  only:
    - main
  tags:
    - K8s-Runner
```

### 4. BuildKit deployment

A new instruction set and image format from Docker, but still based on DinD or DooD operationally.

BuildKit has two common modes:

#### Mode 1: Docker (DinD / DooD) + BuildKit

Just set environment variables:

```yaml
variables:
  DOCKER_BUILDKIT: "1"
  BUILDKIT_PROGRESS: plain
```

#### Mode 2: GitLab Runner + BuildKit Pod

* Deploy a buildkitd DaemonSet or Deployment in K8s
* Each runner job uses the buildctl CLI to call the in-cluster buildkitd service to build
* No need to mount `/var/run/docker.sock` — security higher than DinD

![Kubernetes BuildKit container image build flow from GitLab Runner](https://blog.markkulab.net/content/markku/posts/k8s-build-image-methods/images/buildKit-pod.png)

## Deployment recommendations

### When to use each

1. **Small-team quick CI**: use DinD
2. **Internal-use CI**: use DooD
3. **Cloud-native CI/CD**: use Kaniko
4. **Need caching/performance**: use BuildKit
5. **OpenShift/Red Hat systems**: use Buildah

### Security considerations

* **DinD and DooD** require privileged mode — lower security
* **Kaniko and BuildKit** don't require privileged mode — higher security
* For production, recommend Kaniko or BuildKit

> **Advanced: moby/BuildKit** has more complex setup, which I haven't researched. Requires deploying buildkitd Pods — both based on DinD / DooD operationally.

## Conclusion

Configuring these environments is troublesome — small differences in environment can produce different behavior. Knowing several build methods means you can handle different environments. As time and tech iterate, these issues should diminish.

Actually, the cleaner approach is splitting CI and CD — safer, simpler config, no K8s-CI compatibility errors.

## References

* [GitLab Runner can't call host's Docker](https://johnnyexplores.medium.com/gitlab-runner%E7%84%A1%E6%B3%95%E5%91%BC%E5%8F%ABhost%E7%9A%84docker-e82dd3f5ae27)

---

## About this article and its author

Originally published on [Mark Ku's Tech Notes](https://blog.markkulab.net/en/post/k8s-build-image-methods)

License: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) — when reusing or quoting, credit the author and link back to the original

### About the author

**[Mark Ku](https://blog.markkulab.net/en/author/mark-ku)** — Software Solution Provider

- 10+ years senior software engineer, now an AI Builder
- Focused on large-platform architecture — North-American e-commerce, AI SaaS subscription billing
- Combining AI Agents and automation to build evolvable product foundations

### Free tools built by the author

All of these are free to use:

- [Free PDF Sign Tool](https://blog.markkulab.net/en/tools/pdf-sign): Online PDF sign tool — draw, type, or upload a signature, then drag, resize, and download. Everything runs in your browser; nothing is uploaded.
- [VS Code Refactory](https://blog.markkulab.net/en/tools/refactory): Refactory is a VS Code refactoring extension: 34 actions plus a 37-rule code-smell inspection layer with a Code Health dashboard, across 18 languages, backed by 534 tests. It learns your repo's conventions: where interfaces live, where DI is registered, whether 'use client' belongs. It ranks files by git churn × complexity so you know what to fix first, and hands any smell to the Claude Code already on your machine. Free to use, and your source never leaves your computer.
- [DB-Kit Database Manager](https://blog.markkulab.net/en/tools/db-kit): DB-Kit is a lightweight, cross-platform database manager built with Tauri + Rust + React. Manage MySQL, MariaDB, PostgreSQL, SQL Server, Oracle, SQLite, MongoDB, Redis, Kafka, Elasticsearch and RabbitMQ from one consistent interface: passwords encrypted in the OS keychain, SSH tunnels, full CRUD, a visual query builder, stacked multi-statement result sets, cross-connection data transfer and compare/sync, Excel / CSV import & export, visualized execution plans, ER diagrams, scheduled backups, SQL stress testing with p50–p99 latency percentiles, a 15-rule SQL review engine, Kafka message browsing with monitoring & alerts, a bilingual UI (Traditional Chinese / English), a built-in AI assistant (natural-language SQL, AI review and tuning advice) and the dbk CLI. Free and open source (MIT), with installers for Windows, macOS and Linux.
- [VS Code Super Mermaid](https://blog.markkulab.net/en/tools/super-mermaid): Super Mermaid is a VS Code extension for beautiful Mermaid diagrams out of the box: auto-colored live preview, mouse pan & zoom, high-res PNG / SVG export, 21 templates and multiple themes. Free and open source (MIT).
- [React Super Mermaid](https://blog.markkulab.net/en/tools/react-super-mermaid): react-super-mermaid is an open-source React component library: render beautiful Mermaid diagrams with a single <MermaidViewer>, with built-in colorful / sketch themes, pan & zoom, in-diagram search, and high-res SVG / PNG export. Lightweight, SSR-safe, fully typed. Free and open source (MIT).
- [Jira / Confluence Super Mermaid](https://blog.markkulab.net/en/tools/jira-super-mermaid): An Atlassian Forge app: write Mermaid syntax directly inside a Jira issue or a Confluence page and get flowcharts, sequence diagrams, state machines and Gantt charts. 11 diagram types, SVG / PNG export, light and dark themes, full CJK support. Runs on Atlassian: your diagrams live in your own site and the app calls no third-party service. Free, coming soon to the Atlassian Marketplace.
- [Mermaid Live Preview](https://blog.markkulab.net/en/tools/mermaid-preview): Write Mermaid in your browser, see it render instantly, and share the whole diagram as a single link. No sign-up, nothing uploaded to a server, and mermaid.live share links work as-is.
- [React Intl Phone Number](https://blog.markkulab.net/en/tools/react-intl-phone-number): react-intl-phone-number is an open-source React component: framework-agnostic and antd-free, with E.164 in/out, a searchable flag / country-code dropdown, configurable validation levels (strict / mobile-strict / loose), themeable CSS, and i18n — phone logic powered by google-libphonenumber. Lightweight and fully typed. Free and open source (MIT).
- [Uptime Kuma Cluster](https://blog.markkulab.net/en/tools/uptime-kuma-cluster): Turn single-node Uptime Kuma into a highly available cluster: OpenResty + Lua smart load balancing, shared MariaDB state, health checks and automatic failover, plus cluster-management REST APIs. One Docker Compose command to start. Free and open source (MIT).
- [Special Education](https://blog.markkulab.net/en/education): Learning materials crafted for special education students

### Daily podcasts

- [Mark's Tech Insights — Daily AI News](https://blog.markkulab.net/en/category/tech-news): Daily curated AI and tech trends. Catch the latest developments via audio summaries — covering AI applications, software architecture, DevOps, and engineering practice. — RSS: https://blog.markkulab.net/feed.xml
- [AI股市蝦聊](https://blog.markkulab.net/en/category/ai-stock-chat): Every trading day, an AI-analyzed take on the Taiwan stock market, delivered as a two-host conversation covering the session and the next-day outlook. — RSS: https://blog.markkulab.net/ai-stock-chat/feed.xml
- [開源好物週報](https://blog.markkulab.net/en/category/open-source-weekly): A weekly two-host pick of free open-source tools surfaced from real Hacker News, GitHub, and Reddit buzz — what pain they solve and the fastest way to get started. — RSS: https://blog.markkulab.net/open-source-weekly/feed.xml

### Newsletter

[Subscribe to the newsletter](https://blog.markkulab.net/en/subscribe) — Be the first to know about new posts. No spam, unsubscribe anytime.