DockerCurrent Docker Engine / ComposeINTERMEDIATE

Docker Containers & Compose

Beginner-to-advanced Docker reference covering images, containers, Dockerfiles, BuildKit, cache, lifecycle, storage, networking, Compose, registries, multi-platform builds, security, diagnostics, and production practices.

62 min read

Category: DevOps

Images, Containers & Lifecycle

Distinguish immutable image content from runtime process and writable state.

Image, Container & Registry Mental Model

An image is a content-addressed layer/config template; a container is an isolated process using that image plus runtime configuration and a writable layer; a registry distributes images.

Visual Architecture & Flowchart

Mermaid.js

flowchart LR D[Dockerfile + context] --> B[BuildKit] B --> I[Immutable image layers] I --> R[Registry] I --> C1[Container process + writable layer] I --> C2[Container process + writable layer] V[Volume] --> C1

🛑

Containers Are Disposable

A container's writable layer disappears with removal. Persist application data in volumes or external services and design processes to restart safely.

run, create, start, stop & remove

run creates and starts; create separates configuration from start; stop sends the configured stop signal then kills after timeout; rm deletes stopped containers.

bash

docker run --name web --detach --publish 8080:8080 --env-file .env app:1.4

docker ps --all

docker stop --time 30 web

docker start web

docker rm web

# Temporary interactive tool

docker run --rm -it --mount type=bind,src="$PWD",dst=/work -w /work alpine:3.22 sh

⚠️

PID 1 Must Handle Signals

Use exec-form ENTRYPOINT/CMD so the application receives SIGTERM. Avoid shell wrappers that fail to exec or reap children; use --init when a tiny init is appropriate.

inspect, exec, logs & stats

Inspect declared/runtime state, execute short diagnostics in a running container, stream logs, and observe resource usage.

bash

docker inspect web

docker logs --follow --tail 100 --timestamps web

docker exec -it web sh

docker top web

docker stats --no-stream web

docker port web

docker diff web

🧯

Do Not Repair Containers Manually

exec changes are unrepeatable and disappear after replacement. Diagnose, then fix the image/configuration and redeploy; never treat a running container like a hand-maintained VM.

Dockerfiles & BuildKit

Build reproducible minimal images with intentional cache and secret boundaries.

FROM, WORKDIR, COPY, RUN, USER & CMD

Each instruction contributes config or a filesystem layer. Use a pinned, maintained base; stable workdir; narrow COPY; non-root runtime; and one foreground process.

dockerfile

# syntax=docker/dockerfile:1

FROM node:24-alpine AS runtime

WORKDIR /app

ENV NODE_ENV=production

COPY --chown=node:node package.json package-lock.json ./

RUN npm ci --omit=dev && npm cache clean --force

COPY --chown=node:node dist ./dist

USER node

EXPOSE 8080

CMD ["node", "dist/server.js"]

📌

EXPOSE Is Documentation

It does not publish a host port. Use docker run -p or Compose ports; bind the application to 0.0.0.0 inside the container when it must accept external traffic.

Layer Cache & .dockerignore

Build cache reuses an instruction when its inputs match; once invalidated, following steps rebuild. Copy dependency manifests before frequently changing source.

dockerfile

COPY package.json package-lock.json ./

RUN --mount=type=cache,target=/root/.npm npm ci

COPY . .

RUN npm run build

🔐

Context Is Data Sent to the Builder

Exclude secrets, VCS history, dependencies, and irrelevant artifacts. COPY . . can leak files into layers even if later deleted.

Multi-Stage Builds, Secrets & Cache Mounts

Build in a tool-rich stage and copy only runtime artifacts. BuildKit mounts provide temporary cache, secret, SSH, bind, or tmpfs data without committing it to layers.

dockerfile

# syntax=docker/dockerfile:1

FROM node:24 AS build

WORKDIR /src

COPY package*.json ./

RUN --mount=type=cache,target=/root/.npm npm ci

COPY . .

RUN --mount=type=secret,id=npmrc,target=/root/.npmrc npm run build

FROM node:24-alpine

WORKDIR /app

COPY --from=build --chown=node:node /src/dist ./dist

USER node

CMD ["node", "dist/server.js"]

🔑

ARG and ENV Are Not Secret Storage

They can appear in metadata, history, cache, logs, or final config. Use secret mounts during build and runtime secret injection outside the image.

Storage & Networking

Choose explicit persistence, sharing, and connectivity contracts.

Volumes, Bind Mounts & tmpfs

Named volumes are Docker-managed persistent data; bind mounts expose host paths; tmpfs keeps temporary data in memory.

Container Storage Selection

Mount Type	Data Location	Persists	Best For	Main Risk
Named volume	Docker-managed host storage	After container removal	Databases and application data	Must be backed up and lifecycle-managed separately
Bind mount	Explicit host path	As long as host files exist	Source code, local config, host integration	Host coupling and accidental file exposure
tmpfs	Host memory	No	Secrets or high-churn temporary data	Consumes memory and vanishes on stop
Container writable layer	Container storage driver	Only until container removal	Disposable runtime changes	Poor choice for important or high-write data

bash

docker volume create postgres-data

docker run --mount type=volume,src=postgres-data,dst=/var/lib/postgresql/data postgres:17

docker run --mount type=bind,src="$PWD/src",dst=/app/src,readonly app-dev

docker run --mount type=tmpfs,dst=/run/secrets,tmpfs-mode=0700 app

⚠️

Mounts Override Image Paths

Mounting over a non-empty container directory hides image files. Define ownership/UID, backup/restore, migration, and removal policy for named volumes.

Networks, DNS & Port Publishing

User-defined bridge networks provide service-name DNS and isolation. Container ports are internal; published ports bind host interfaces.

bash

docker network create app-net

docker run -d --name db --network app-net postgres:17

docker run -d --name api --network app-net -p 127.0.0.1:8080:8080 app

# API connects to db:5432, not localhost:5432.

🌐

localhost Means This Container

Use the other container's DNS/service name. Bind published development ports to 127.0.0.1 unless LAN exposure is intended.

Resource Limits, Health & Restart

Set memory/CPU/process limits, application-aware health checks, stop grace, and restart policy according to workload behavior.

bash

docker run -d --name api \

--memory 512m --cpus 1.0 --pids-limit 200 \

--health-cmd='wget -qO- http://127.0.0.1:8080/health || exit 1' \

--health-interval=30s --health-timeout=3s --health-retries=3 \

--restart unless-stopped app:1.4

⚠️

Restart Is Not Recovery

A crash loop can amplify dependency load and hide corruption. Emit diagnostics, use backoff/orchestration, make startup idempotent, and distinguish alive from ready.

Docker Compose

Define a trusted multi-container application with reproducible service, network, volume, and dependency configuration.

Services, Networks, Volumes & Environment

Compose services describe container configuration. Service names are network DNS names; named volumes persist; environment interpolation and container environment are distinct.

yaml

services:

api:

build: { context: ., target: runtime }

ports: ["127.0.0.1:8080:8080"]

environment:

DATABASE_URL: postgres://app:${DB_PASSWORD}@db/app

depends_on:

db: { condition: service_healthy }

db:

image: postgres:17

environment: { POSTGRES_USER: app, POSTGRES_PASSWORD: ${DB_PASSWORD}, POSTGRES_DB: app }

volumes: ["db-data:/var/lib/postgresql/data"]

healthcheck: { test: ["CMD-SHELL", "pg_isready -U app"], interval: 5s, timeout: 3s, retries: 10 }

volumes: { db-data: {} }

⚠️

depends_on Is Not Application Resilience

Startup ordering/health can reduce races, but services must retry transient dependency loss after startup and shut down safely.

Compose CLI, Profiles & Overrides

Render effective configuration before applying; use profiles for optional tools and layered files carefully for environments.

bash

docker compose config

docker compose up --detach --build --wait

docker compose ps

docker compose logs -f api

docker compose exec api sh

docker compose run --rm api npm test

docker compose --profile debug up

docker compose down

docker compose down --volumes # deletes declared volumes

🔐

Compose Files Are Trusted Code

They can mount host files, request privileges, and expose secrets during interpolation/config rendering. Review remote Compose sources before running them.

Development vs Production Compose

Development may bind source and expose ports; production should use immutable tagged/digest-pinned images, controlled secrets, resource/policy settings, and external observability.

text

Development: build source, bind mounts, watch, debug ports, local dependencies

Production: pull verified image, no source bind, non-root/read-only, secrets, limits, health, logs

Use docker compose config with every file/env combination in CI.

Do not assume local Compose provides cluster scheduling, HA, rolling updates, or durable secret management.

🔗

Orchestration Boundary

See Kubernetes for reconciliation, replicas, rollout, Services, cluster scheduling, policy, and persistent-volume orchestration.

Registries, Platforms & Supply Chain

Publish immutable, verifiable artifacts for the target architecture.

Tags, Digests, Login & Push

Tags are mutable names; digests identify exact content. Authenticate with scoped tokens and promote tested digests rather than rebuilding per environment.

bash

docker login registry.example.com

docker tag app:local registry.example.com/team/app:1.4.2

docker push registry.example.com/team/app:1.4.2

docker image inspect registry.example.com/team/app:1.4.2

docker pull registry.example.com/team/app@sha256:...

🏷️

latest Is Not a Version

Deploy an immutable digest or unique release tag. Mutable tags make rollback, audit, cache, and incident diagnosis ambiguous.

buildx & Multi-Platform Images

Build a manifest list for target architectures using native builders, cross-compilation, or emulation; test runtime-native dependencies on each platform.

bash

docker buildx create --use --name multi

docker buildx build \

--platform linux/amd64,linux/arm64 \

--tag registry.example.com/team/app:1.4.2 \

--provenance=true --sbom=true --push .

⚠️

Emulation Can Hide Problems

QEMU builds may be slow and do not replace tests on target architecture. Pin platform-specific base images and verify native addons/binaries.

SBOM, Provenance, Signing & Scanning

Generate attestations during build, scan packages/config, sign or verify artifacts using your supply-chain system, and define vulnerability policy with ownership.

text

Build from pinned reviewed bases

Generate SBOM + provenance in CI

Scan OS and application dependencies

Sign/attest the pushed digest

Verify before deployment

Rebuild when base/security fixes arrive

Record exceptions with expiry and exploitability context

📌

Scanning Is a Snapshot

New vulnerabilities appear after publication. Continuously rescan deployed digests and maintain a rebuild/patch cadence.

Container Security & Hardening

Minimize privileges, writable surface, secrets, attack tools, and daemon trust.

Non-Root, Capabilities & Read-Only Filesystem

Set a numeric non-root user, drop capabilities, avoid privileged mode/host namespaces/socket mounts, and make root filesystem read-only with explicit writable mounts.

bash

docker run --read-only \

--user 10001:10001 \

--cap-drop ALL \

--security-opt no-new-privileges \

--tmpfs /tmp:rw,noexec,nosuid,size=64m \

app:1.4.2

🛑

Docker Socket Equals Host Control

Mounting /var/run/docker.sock generally grants control equivalent to root on the host. Use purpose-built restricted APIs/builders instead.

Minimal Images & Runtime Secrets

Ship only runtime files and libraries, omit shells/package managers when feasible, and mount/inject secrets at runtime with rotation and access controls.

text

Do not COPY .env, credentials, SSH keys, cloud config, or VCS history

Do not pass secrets via image ARG/ENV or command-line where metadata exposes them

Prefer runtime secret files/agents over broad environment where supported

Never log effective config containing secrets

Separate build credentials from runtime credentials

📌

Smaller Is Not Automatically Safer

Minimal images reduce surface but still need provenance, updates, compatible libraries, and usable diagnostics. Choose maintained bases and test operational needs.

Daemon, Desktop & Host Boundary

Anyone controlling the Docker daemon can usually control the host. Protect its API, groups, contexts, credentials, and build agents.

text

Never expose unauthenticated Docker TCP API

Treat membership in docker group as privileged

Use TLS/SSH contexts for remote engines

Separate untrusted builds and apply quotas

Review bind mounts, device access, host network/PID, privileged, capabilities

Keep Engine/Desktop/host kernel patched

⚠️

Container Isolation Is Not a VM Guarantee

Containers share the host kernel. Use stronger sandbox/VM isolation for hostile multi-tenancy and enforce kernel, runtime, and orchestration security controls.

Diagnostics, Cleanup & Checklist

Debug layer-by-layer and remove only resolved, explicitly targeted resources.

Build & Runtime Troubleshooting

Inspect effective build stages, history, image config, mounts, networks, DNS, logs, health, and process signals.

bash

docker build --progress=plain --target build .

docker history --no-trunc app:1.4.2

docker inspect app:1.4.2

docker inspect web --format '{{json .State.Health}}'

docker network inspect app-net

docker volume inspect db-data

docker events --since 10m

💡

Debug Without Changing the Artifact

Use an ephemeral debug image/container joined to the same network/volumes when the production image intentionally lacks a shell.

Disk Usage & Safe Cleanup

Measure images, layers, containers, volumes, and build cache before pruning; unused volumes may contain the only copy of data.

bash

docker system df --verbose

docker ps -a --filter status=exited

docker image ls --digests

docker volume ls

docker builder prune --filter until=168h

docker image prune

# Review targets before any system/volume prune.

🗑️

Volume Deletion Is Data Deletion

Confirm ownership, backup, and restore before docker volume rm, compose down -v, or volume prune. Prune commands are not routine harmless cleanup.

Production Checklist

Review reproducibility, runtime ownership, persistence, exposure, supply chain, privilege, and operations.

text

□ Base and deployed images are pinned/traceable by digest

□ Multi-stage image contains only runtime artifacts

□ .dockerignore excludes secrets and irrelevant context

□ Build secrets use mounts, never ARG/ENV/layers

□ Process runs non-root and handles SIGTERM as PID 1

□ Writable data uses explicit volumes/external stores

□ Ports bind only intended interfaces

□ CPU, memory, PIDs, health, and stop grace are defined

□ Root filesystem/capabilities/privileges are minimized

□ Logs go to stdout/stderr without secrets

□ Image is scanned, attested, patched, and multi-platform tested

□ Compose effective config is reviewed and dependencies retry

□ Backup/restore and safe rollback are proven

□ Debug and cleanup procedures avoid mutating/deleting data

Related References

Recommended Next Sheets

DevOps / Git

Git Modern Command Reference & Internal Architecture

v2.45

High-density Git cheat sheet covering interactive staging, branch rebase workflows, reflog recovery, and visual branch architecture.

GITDEVOPSREBASEREFLOGBRANCHING

INTERMEDIATE

12 MIN