marslanalvi avatar

devops-containers-kubernetes

Docker, Kubernetes, Helm, service mesh and GitOps

version
1.0.0
GitHub stars
5

Install

/plugin install devops-containers-kubernetes@skills-stack

Run it inside Claude Code after adding the marketplace with /plugin marketplace add marslanalvi/SkillsStack.

Contents

Discovered from source

Skills 11

Container Security Hardening Skill

A production-focused guide for building, scanning, and running containers securely — from Dockerfile authoring through runtime enforcement and supply chain integrity.


Detailed Guide

Read the detailed guide before executing this skill. It retains the complete procedure and reference material. Treat its safety, prerequisites, and validation requirements as mandatory. For focused work, load the relevant sections; for end-to-end work, read the guide completely.

When to Use This Skill

  • User mentions Docker security, container hardening, or Dockerfile security review
  • User asks about distroless images, non-root containers, or read-only filesystems
  • User wants to scan images for CVEs with Trivy, Grype, or Snyk
  • User mentions seccomp, AppArmor, Linux capabilities, or runtime security
  • User asks "is my Dockerfile secure?" or "how do I reduce my image attack surface?"
  • User wants to sign/verify images with Cosign or generate SBOMs
  • User asks about Kubernetes pod security, NetworkPolicy, or RBAC hardening
  • User says "fix container CVEs" or "harden my container for production"

When NOT to Use This Skill

  • The user is primarily asking about GitHub Actions CI/CD → recommend github-actions-advanced
  • The user needs general Docker usage help (not security) → recommend docker-expert
  • The user is working with Kubernetes orchestration beyond security → recommend kubernetes-architect
  • The user needs application-level security (SQL injection, XSS) → recommend api-security-best-practices

Security Checklist

Dockerfile

  • Supported minimal base image compatible with the application
  • Multi-stage build — no build tools, devDependencies, or compilers in runtime image
  • Non-root USER declared before CMD/ENTRYPOINT
  • Base image pinned to @sha256:... digest (not just tag)
  • No secrets in ENV, ARG, or RUN commands
  • HEALTHCHECK defined
  • OCI labels present (org.opencontainers.image.*)
  • .dockerignore excludes .git, .env, secrets, tests
  • ENTRYPOINT uses exec form, not shell form

Image Scanning

  • Trivy or Grype scan in CI (fails on HIGH/CRITICAL)
  • Hadolint passes with no warnings
  • Secret scan run on image (trivy --scanners secret)
  • SBOM generated and stored
  • .trivyignore has justified entries for accepted CVEs

Runtime

  • --read-only filesystem
  • --cap-drop ALL (add back only what's documented as required)
  • --security-opt no-new-privileges:true
  • Runtime default seccomp applied; custom profile only after isolated validation
  • Resource limits set (--memory, --cpus, --pids-limit)
  • Image signed with Cosign; verified before deploy

Kubernetes

  • readOnlyRootFilesystem: true
  • allowPrivilegeEscalation: false
  • runAsNonRoot: true with explicit UID
  • capabilities.drop: ["ALL"]
  • Resource requests and limits defined
  • automountServiceAccountToken: false
  • Namespace PSA enforced at restricted level
  • NetworkPolicy default-deny applied
  • RBAC uses specific resource names and minimal verbs

Limitations

  • Cluster-wide admission policies, registry settings and runtime restrictions can disrupt workloads. Review the intended scope, test in staging and provide rollback before applying. Bundled profiles and manifests are examples, not a security certification.

  • Use this skill only when the task clearly matches the scope described above.

  • Do not treat the output as a substitute for environment-specific penetration testing or a formal security audit.

  • Seccomp and AppArmor act on the Linux host/kernel; Docker Desktop may apply them inside its Linux VM depending on runtime support.

  • Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.

Docker Patterns

Docker and Docker Compose best practices for containerized development.

Docker Compose for Local Development

Standard Web App Stack

# docker-compose.yml
services:
  app:
    build:
      context: .
      target: dev                     # Use dev stage of multi-stage Dockerfile
    ports:
      - "3000:3000"
    volumes:
      - .:/app                        # Bind mount for hot reload
      - /app/node_modules             # Anonymous volume -- preserves container deps
    environment:
      - DATABASE_URL=postgres://postgres:postgres@db:5432/app_dev
      - REDIS_URL=redis://redis:6379/0
      - NODE_ENV=development
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_started
    command: npm run dev

  db:
    image: postgres:16-alpine
    ports:
      - "5432:5432"
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: app_dev
    volumes:
      - pgdata:/var/lib/postgresql/data
      - ./scripts/init-db.sql:/docker-entrypoint-initdb.d/init.sql
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 3s
      retries: 5

  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    volumes:
      - redisdata:/data

  mailpit:                            # Local email testing
    image: axllent/mailpit
    ports:
      - "8025:8025"                   # Web UI
      - "1025:1025"                   # SMTP

volumes:
  pgdata:
  redisdata:

Development vs Production Dockerfile

# Stage: dependencies
FROM node:22-alpine AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci

# Stage: dev (hot reload, debug tools)
FROM node:22-alpine AS dev
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
EXPOSE 3000
CMD ["npm", "run", "dev"]

# Stage: build
FROM node:22-alpine AS build
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN npm run build && npm prune --production

# Stage: production (minimal image)
FROM node:22-alpine AS production
WORKDIR /app
RUN addgroup -g 1001 -S appgroup && adduser -S appuser -u 1001
USER appuser
COPY --from=build --chown=appuser:appgroup /app/dist ./dist
COPY --from=build --chown=appuser:appgroup /app/node_modules ./node_modules
COPY --from=build --chown=appuser:appgroup /app/package.json ./
ENV NODE_ENV=production
EXPOSE 3000
HEALTHCHECK --interval=30s --timeout=3s CMD wget -qO- http://localhost:3000/health || exit 1
CMD ["node", "dist/server.js"]

Override Files

# docker-compose.override.yml (auto-loaded, dev-only settings)
services:
  app:
    environment:
      - DEBUG=app:*
      - LOG_LEVEL=debug
    ports:
      - "9229:9229"                   # Node.js debugger

# docker-compose.prod.yml (explicit for production)
services:
  app:
    build:
      target: production
    restart: always
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 512M
# Development (auto-loads override)
docker compose up

# Production
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

Networking

Service Discovery

Services in the same Compose network resolve by service name:

# From "app" container:
postgres://postgres:postgres@db:5432/app_dev    # "db" resolves to the db container
redis://redis:6379/0                             # "redis" resolves to the redis container

Custom Networks

services:
  frontend:
    networks:
      - frontend-net

  api:
    networks:
      - frontend-net
      - backend-net

  db:
    networks:
      - backend-net              # Only reachable from api, not frontend

networks:
  frontend-net:
  backend-net:

Exposing Only What's Needed

services:
  db:
    ports:
      - "127.0.0.1:5432:5432"   # Only accessible from host, not network
    # Omit ports entirely in production -- accessible only within Docker network

Volume Strategies

volumes:
  # Named volume: persists across container restarts, managed by Docker
  pgdata:

  # Bind mount: maps host directory into container (for development)
  # - ./src:/app/src

  # Anonymous volume: preserves container-generated content from bind mount override
  # - /app/node_modules

Common Patterns

services:
  app:
    volumes:
      - .:/app                   # Source code (bind mount for hot reload)
      - /app/node_modules        # Protect container's node_modules from host
      - /app/.next               # Protect build cache

  db:
    volumes:
      - pgdata:/var/lib/postgresql/data          # Persistent data
      - ./scripts/init.sql:/docker-entrypoint-initdb.d/init.sql  # Init scripts

Container Security

Dockerfile Hardening

# 1. Use specific tags (never :latest)
FROM node:22.12-alpine3.20

# 2. Run as non-root
RUN addgroup -g 1001 -S app && adduser -S app -u 1001
USER app

# 3. Drop capabilities (in compose)
# 4. Read-only root filesystem where possible
# 5. No secrets in image layers

Compose Security

services:
  app:
    security_opt:
      - no-new-privileges:true
    read_only: true
    tmpfs:
      - /tmp
      - /app/.cache
    cap_drop:
      - ALL
    cap_add:
      - NET_BIND_SERVICE          # Only if binding to ports < 1024

Secret Management

# GOOD: Use environment variables (injected at runtime)
services:
  app:
    env_file:
      - .env                     # Never commit .env to git
    environment:
      - API_KEY                  # Inherits from host environment

# GOOD: Docker secrets (Swarm mode)
secrets:
  db_password:
    file: ./secrets/db_password.txt

services:
  db:
    secrets:
      - db_password

# BAD: Hardcoded in image
# ENV API_KEY=sk-proj-xxxxx      # NEVER DO THIS

Hardened CLI Installer Harnesses

Use containers to test installer behavior against disposable project copies without allowing the test to mutate the source checkout.

Respect the Platform Boundary

  • Run real containers for Linux distributions such as Debian and Ubuntu.
  • macOS cannot run as a Docker container because Docker shares a Linux kernel. Run the same shell-free test entry point natively on macOS.
  • Windows containers require a Windows Docker engine. Run platform-independent logic on a native Windows CI runner and reserve Windows containers for a Windows host.
  • Keep a native Ubuntu/macOS/Windows CI matrix for host-specific paths, command shims, quoting, and filesystem behavior.

Do not claim that a Linux container validates macOS or Windows behavior.

Enforce the Isolation Contract

  • Pin base images by immutable digest and pin installed CLI versions.
  • Run as a non-root numeric UID/GID when distro account names differ.
  • Mount the repository and source project read-only.
  • Copy the source project into a writable tmpfs workspace before any mutation.
  • Mount /workspace with noexec, UID/GID 1000, and mode=0700 so only the container user can inspect project data.
  • Keep npm and npx's executable cache at NPM_CONFIG_CACHE=/tmp/npm-cache on the executable /tmp mount. Its default size is 2 GiB and can be adjusted with ECC_TMPFS_SIZE; ECC_WORKSPACE_SIZE separately controls the private workspace mount.
  • Set read_only: true, no-new-privileges:true, cap_drop: [ALL], and a finite pids_limit.
  • Keep the default real-CLI services on network_mode: none. Add network access only through a visibly named opt-in service for an authenticated provider session; never make it an accidental environment-driven default.
  • Create only the writable temporary paths the tool needs.
  • Do not pass host credentials into the container by default.
  • Default to a dry run and whitelist only the explicit dry-run, install, plugin, and shell modes.
  • Use argument arrays or spawnSync(..., { shell: false }) for cross-platform runners. Never interpolate project paths into a shell command.

Exercise the ECC Plugin Setup Harness

Use docker/plugin-setup/compose.yaml as the reference implementation. It provides:

  • fixture-tests for the focused install manifest, target, and executor suite.
  • real-cli for the pinned Debian-based generic Linux image.
  • real-cli-ubuntu for the pinned Ubuntu image.

Validate the Compose model before building:

docker compose -f docker/plugin-setup/compose.yaml config --quiet

Build both real Linux images:

docker compose -f docker/plugin-setup/compose.yaml \
  build real-cli real-cli-ubuntu

Run the safe default flow in each image:

docker compose -p ecc-plugin-debian-test \
  -f docker/plugin-setup/compose.yaml \
  run --rm -T real-cli dry-run

docker compose -p ecc-plugin-ubuntu-test \
  -f docker/plugin-setup/compose.yaml \
  run --rm -T real-cli-ubuntu dry-run

The dry run executes the current public command contract:

ecc install --profile core --target claude-project --dry-run --json

Before that command runs, the container creates a locally packed npm artifact from the read-only checkout with npm pack --ignore-scripts. It extracts the self-created tarball under /tmp, validates the ecc-universal package name, required install manifests, and the confined package.json bin.ecc mapping, then invokes the extracted ecc executable. The runtime stays on network_mode: none, does not execute package lifecycle scripts, and does not rely on host node_modules; its exact pinned production dependencies are already present in the image.

The harness rejects an empty plan, a non-claude-project target, any operation outside /workspace/project/.claude, or any dry run that creates the target directory. install performs the isolated apply twice, checks its managed install state, lists the installed target, and runs doctor.

Start, Open, Reconnect, and Clean Up a Named Session

Start a detached container without --rm so leaving a terminal does not remove the session:

docker compose -p ecc-plugin-session \
  -f docker/plugin-setup/compose.yaml \
  run --detach --name ecc-plugin-shell real-cli shell

The container copies the read-only fixture to the stable private directory /workspace/project. Confirm it is running, then emit the Docker side of the terminal-opener v1 data contract:

docker inspect --format '{{.State.Running}}' ecc-plugin-shell
node docker/plugin-setup/interactive-plan.js \
  --container ecc-plugin-shell \
  --workdir /workspace/project \
  --json \
  -- bash

The JSON result has exactly an executable and argv boundary (plus contractVersion: 1): the executable is docker, and argv begins with exec, -it, and -w. Pass that data to the separate terminal-opener skill when it is installed. This Docker harness deliberately does not import a terminal adapter, interpolate a shell command, or manage a host GUI process. Until then, open the same PTY in the current host terminal directly:

docker exec -it -w /workspace/project ecc-plugin-shell bash

Exit the shell without stopping the detached container. Reconnect with the same docker exec -it command. When finished, remove the exact named container and its Compose project resources:

docker rm --force ecc-plugin-shell
docker compose -p ecc-plugin-session \
  -f docker/plugin-setup/compose.yaml \
  down --remove-orphans

Host credentials are absent by default and credential directories are never mounted. The default service also has no network access. When an authenticated provider session genuinely needs a network, build real-cli first and then opt in visibly with docker compose -f docker/plugin-setup/compose.yaml --profile networked run --service-ports real-cli-networked shell. The --service-ports flag publishes the service's loopback callback mapping. Prefer authenticating inside that disposable session. If a CI run must inherit a host environment credential, make that opt-in at invocation with an explicit Compose --env NAME flag, understand that the value is inspectable and can be exfiltrated for the container lifetime, and remove the exact named container immediately after.

Authenticate Codex Inside the Networked Session

codex login binds its OAuth callback server to 127.0.0.1:1455 inside the container. A Docker port publish forwards to the container interface, not loopback, so the host browser's redirect to http://localhost:1455 is refused without a bridge. The real-cli-networked service publishes host 127.0.0.1:1455 to container port 1456; bridge that port to loopback before logging in:

(
  set -e
  npm install --global @openai/codex@0.162.0   # the harness image does not ship codex
  node /ecc/docker/plugin-setup/codex-auth-relay.js &
  relay_pid=$!
  trap 'kill "$relay_pid" 2>/dev/null || true; wait "$relay_pid" 2>/dev/null || true' EXIT
  codex login
)

The networked service sets NPM_CONFIG_PREFIX=/tmp/ecc-npm and adds its bin directory to PATH. This keeps the pinned installation writable for UID 1000 on the executable /tmp mount while the root filesystem remains read-only.

Complete the sign-in in the host browser. Each attempt mints a fresh state/code, so always follow the newest printed link. The token stays in the disposable container's ~/.codex/auth.json; removing the named container deletes that credential copy. The login block stops the relay on success or failure. Only one container can hold host port 1455 at a time because the OAuth redirect targets a fixed port.

The host publish is restricted to loopback. Inside the container, the relay listens on all interfaces so Docker can forward to it; other containers on the same bridge network can reach it while it runs. Use a dedicated network without untrusted peers for this login, and remove the exact named container after use. Idle callback connections close after 30 seconds.

Run the same focused suite natively on the host:

npm run test:plugin-setup-platform

Inspect the produced identity and environment before trusting the image:

docker image inspect ecc-plugin-setup:debian ecc-plugin-setup:ubuntu

Clean each named test project without deleting unrelated volumes or images:

docker compose -p ecc-plugin-debian-test \
  -f docker/plugin-setup/compose.yaml down --remove-orphans
docker compose -p ecc-plugin-ubuntu-test \
  -f docker/plugin-setup/compose.yaml down --remove-orphans

.dockerignore

node_modules
.git
.env
.env.*
dist
coverage
*.log
.next
.cache
docker-compose*.yml
Dockerfile*
README.md
tests/

Debugging

Common Commands

# View logs
docker compose logs -f app           # Follow app logs
docker compose logs --tail=50 db     # Last 50 lines from db

# Execute commands in running container
docker compose exec app sh           # Shell into app
docker compose exec db psql -U postgres  # Connect to postgres

# Inspect
docker compose ps                     # Running services
docker compose top                    # Processes in each container
docker stats                          # Resource usage

# Rebuild
docker compose up --build             # Rebuild images
docker compose build --no-cache app   # Force full rebuild

# Clean up
docker compose down                   # Stop and remove containers
docker compose down -v                # Also remove volumes (DESTRUCTIVE)
docker system prune                   # Remove unused images/containers

Debugging Network Issues

# Check DNS resolution inside container
docker compose exec app nslookup db

# Check connectivity
docker compose exec app wget -qO- http://api:3000/health

# Inspect network
docker network ls
docker network inspect <project>_default

Anti-Patterns

# BAD: Using docker compose in production without orchestration
# Use Kubernetes, ECS, or Docker Swarm for production multi-container workloads

# BAD: Storing data in containers without volumes
# Containers are ephemeral -- all data lost on restart without volumes

# BAD: Running as root
# Always create and use a non-root user

# BAD: Using :latest tag
# Pin to specific versions for reproducible builds

# BAD: One giant container with all services
# Separate concerns: one process per container

# BAD: Putting secrets in docker-compose.yml
# Use .env files (gitignored) or Docker secrets

GitOps Workflow

Complete guide to implementing GitOps workflows with ArgoCD and Flux for automated Kubernetes deployments.

Purpose

Implement declarative, Git-based continuous delivery for Kubernetes using ArgoCD or Flux CD, following OpenGitOps principles.

When to Use This Skill

  • Set up GitOps for Kubernetes clusters
  • Automate application deployments from Git
  • Implement progressive delivery strategies
  • Manage multi-cluster deployments
  • Configure automated sync policies
  • Set up secret management in GitOps

OpenGitOps Principles

  1. Declarative - Entire system described declaratively
  2. Versioned and Immutable - Desired state stored in Git
  3. Pulled Automatically - Software agents pull desired state
  4. Continuously Reconciled - Agents reconcile actual vs desired state

ArgoCD Setup

1. Installation

# Create namespace
kubectl create namespace argocd

# Install ArgoCD
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# Get admin password
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d

Reference: See references/argocd-setup.md for detailed setup

2. Repository Structure

gitops-repo/
├── apps/
│   ├── production/
│   │   ├── app1/
│   │   │   ├── kustomization.yaml
│   │   │   └── deployment.yaml
│   │   └── app2/
│   └── staging/
├── infrastructure/
│   ├── ingress-nginx/
│   ├── cert-manager/
│   └── monitoring/
└── argocd/
    ├── applications/
    └── projects/

3. Create Application

# argocd/applications/my-app.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-app
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/org/gitops-repo
    targetRevision: main
    path: apps/production/my-app
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

4. App of Apps Pattern

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: applications
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/org/gitops-repo
    targetRevision: main
    path: argocd/applications
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  syncPolicy:
    automated: {}

Flux CD Setup

1. Installation

The example uses Homebrew. For other systems, follow the official Flux installation guide.

# Install the Flux CLI with Homebrew
brew install fluxcd/tap/flux
flux check --pre

# Bootstrap Flux
flux bootstrap github \
  --owner=org \
  --repository=gitops-repo \
  --branch=main \
  --path=clusters/production \
  --personal

2. Create GitRepository

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: my-app
  namespace: flux-system
spec:
  interval: 1m
  url: https://github.com/org/my-app
  ref:
    branch: main

3. Create Kustomization

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: my-app
  namespace: flux-system
spec:
  interval: 5m
  path: ./deploy
  prune: true
  sourceRef:
    kind: GitRepository
    name: my-app

Sync Policies

Auto-Sync Configuration

ArgoCD:

syncPolicy:
  automated:
    prune: true # Delete resources not in Git
    selfHeal: true # Reconcile manual changes
    allowEmpty: false
  retry:
    limit: 5
    backoff:
      duration: 5s
      factor: 2
      maxDuration: 3m

Flux:

spec:
  interval: 1m
  prune: true
  wait: true
  timeout: 5m

Reference: See references/sync-policies.md

Progressive Delivery

Canary Deployment with ArgoCD Rollouts

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: my-app
spec:
  replicas: 5
  strategy:
    canary:
      steps:
        - setWeight: 20
        - pause: { duration: 1m }
        - setWeight: 50
        - pause: { duration: 2m }
        - setWeight: 100

Blue-Green Deployment

strategy:
  blueGreen:
    activeService: my-app
    previewService: my-app-preview
    autoPromotionEnabled: false

Secret Management

External Secrets Operator

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-credentials
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secrets-manager
    kind: SecretStore
  target:
    name: db-credentials
  data:
    - secretKey: password
      remoteRef:
        key: prod/db/password

Sealed Secrets

# Encrypt secret
kubeseal --format yaml < secret.yaml > sealed-secret.yaml

# Commit sealed-secret.yaml to Git

Best Practices

  1. Use separate repos or branches for different environments
  2. Implement RBAC for Git repositories
  3. Enable notifications for sync failures
  4. Use health checks for custom resources
  5. Implement approval gates for production
  6. Keep secrets out of Git (use External Secrets)
  7. Use App of Apps pattern for organization
  8. Tag releases for easy rollback
  9. Monitor sync status with alerts
  10. Test changes in staging first

Troubleshooting

Sync failures:

argocd app get my-app
argocd app sync my-app --prune

Out of sync status:

argocd app diff my-app
argocd app sync my-app --force

Related Skills

  • k8s-manifest-generator - For creating manifests
  • helm-chart-scaffolding - For packaging applications

Helm Chart Scaffolding

Comprehensive guidance for creating, organizing, and managing Helm charts for packaging and deploying Kubernetes applications.

Use this skill when

Use this skill when you need to:

  • Create new Helm charts from scratch
  • Package Kubernetes applications for distribution
  • Manage multi-environment deployments with Helm
  • Implement templating for reusable Kubernetes manifests
  • Set up Helm chart repositories
  • Follow Helm best practices and conventions

Do not use this skill when

  • The task is unrelated to helm chart scaffolding
  • You need a different domain or tool outside this scope

Scope and deployment limits

Pin Helm, Kubernetes, image and dependency versions; historical Bitnami versions below are examples, not supported/security-reviewed recommendations. The bundled values/Chart assets and snippets are not a complete chart. Helpers, manifests, schemas and required Secrets must be supplied and rendered consistently. Review dependencies and Chart.lock before downloads. Local lint/template cannot prove API admission or runtime health; server validation requires an authorized cluster/context, namespace and target version. Dry-run/debug output may expose Secrets. Apply/install/upgrade/test hooks and repository publication require explicit target authorization, backup/rollback planning and cost/impact review. Never package credential files in a chart.

Instructions

  • Clarify goals, constraints, and required inputs.
  • Apply relevant best practices and validate outcomes.
  • Provide actionable steps and verification.
  • If detailed examples are required, open resources/implementation-playbook.md.

Resources

  • resources/implementation-playbook.md for detailed patterns and examples.

Limitations

  • Use this skill only when the task clearly matches the scope described above.
  • Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
  • Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.

Istio Traffic Management

Comprehensive guide to Istio traffic management for production service mesh deployments.

When to Use This Skill

  • Configuring service-to-service routing
  • Implementing canary or blue-green deployments
  • Setting up circuit breakers and retries
  • Load balancing configuration
  • Traffic mirroring for testing
  • Fault injection for chaos engineering

Core Concepts

1. Traffic Management Resources

ResourcePurposeScope
VirtualServiceRoute traffic to destinationsHost-based
DestinationRuleDefine policies after routingService-based
GatewayConfigure ingress/egressCluster edge
ServiceEntryAdd external servicesMesh-wide

2. Traffic Flow

Client → Gateway → VirtualService → DestinationRule → Service
                   (routing)        (policies)        (pods)

Templates

Template 1: Basic Routing

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: reviews-route
  namespace: bookinfo
spec:
  hosts:
    - reviews
  http:
    - match:
        - headers:
            end-user:
              exact: jason
      route:
        - destination:
            host: reviews
            subset: v2
    - route:
        - destination:
            host: reviews
            subset: v1
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: reviews-destination
  namespace: bookinfo
spec:
  host: reviews
  subsets:
    - name: v1
      labels:
        version: v1
    - name: v2
      labels:
        version: v2
    - name: v3
      labels:
        version: v3

Template 2: Canary Deployment

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: my-service-canary
spec:
  hosts:
    - my-service
  http:
    - route:
        - destination:
            host: my-service
            subset: stable
          weight: 90
        - destination:
            host: my-service
            subset: canary
          weight: 10
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: my-service-dr
spec:
  host: my-service
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 100
      http:
        h2UpgradePolicy: UPGRADE
        http1MaxPendingRequests: 100
        http2MaxRequests: 1000
  subsets:
    - name: stable
      labels:
        version: stable
    - name: canary
      labels:
        version: canary

Template 3: Circuit Breaker

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: circuit-breaker
spec:
  host: my-service
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 100
      http:
        http1MaxPendingRequests: 100
        http2MaxRequests: 1000
        maxRequestsPerConnection: 10
        maxRetries: 3
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 30s
      baseEjectionTime: 30s
      maxEjectionPercent: 50
      minHealthPercent: 30

Template 4: Retry and Timeout

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: ratings-retry
spec:
  hosts:
    - ratings
  http:
    - route:
        - destination:
            host: ratings
      timeout: 10s
      retries:
        attempts: 3
        perTryTimeout: 3s
        retryOn: connect-failure,refused-stream,unavailable,cancelled,retriable-4xx,503
        retryRemoteLocalities: true

Template 5: Traffic Mirroring

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: mirror-traffic
spec:
  hosts:
    - my-service
  http:
    - route:
        - destination:
            host: my-service
            subset: v1
      mirror:
        host: my-service
        subset: v2
      mirrorPercentage:
        value: 100.0

Template 6: Fault Injection

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: fault-injection
spec:
  hosts:
    - ratings
  http:
    - fault:
        delay:
          percentage:
            value: 10
          fixedDelay: 5s
        abort:
          percentage:
            value: 5
          httpStatus: 503
      route:
        - destination:
            host: ratings

Template 7: Ingress Gateway

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: my-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
    - port:
        number: 443
        name: https
        protocol: HTTPS
      tls:
        mode: SIMPLE
        credentialName: my-tls-secret
      hosts:
        - "*.example.com"
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: my-vs
spec:
  hosts:
    - "api.example.com"
  gateways:
    - my-gateway
  http:
    - match:
        - uri:
            prefix: /api/v1
      route:
        - destination:
            host: api-service
            port:
              number: 8080

Load Balancing Strategies

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: load-balancing
spec:
  host: my-service
  trafficPolicy:
    loadBalancer:
      simple: ROUND_ROBIN # or LEAST_CONN, RANDOM, PASSTHROUGH
---
# Consistent hashing for sticky sessions
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: sticky-sessions
spec:
  host: my-service
  trafficPolicy:
    loadBalancer:
      consistentHash:
        httpHeaderName: x-user-id
        # or: httpCookie, useSourceIp, httpQueryParameterName

Best Practices

Do's

  • Start simple - Add complexity incrementally
  • Use subsets - Version your services clearly
  • Set timeouts - Always configure reasonable timeouts
  • Enable retries - But with backoff and limits
  • Monitor - Use Kiali and Jaeger for visibility

Don'ts

  • Don't over-retry - Can cause cascading failures
  • Don't ignore outlier detection - Enable circuit breakers
  • Don't mirror to production - Mirror to test environments
  • Don't skip canary - Test with small traffic percentage first

Debugging Commands

# Check VirtualService configuration
istioctl analyze

# View effective routes
istioctl proxy-config routes deploy/my-app -o json

# Check endpoint discovery
istioctl proxy-config endpoints deploy/my-app

# Debug traffic
istioctl proxy-config log deploy/my-app --level debug

Kubernetes Manifest Generator

Step-by-step guidance for creating production-ready Kubernetes manifests including Deployments, Services, ConfigMaps, Secrets, and PersistentVolumeClaims.

Purpose

This skill provides comprehensive guidance for generating well-structured, secure, and production-ready Kubernetes manifests following cloud-native best practices and Kubernetes conventions.

When to Use This Skill

Use this skill when you need to:

  • Create new Kubernetes Deployment manifests
  • Define Service resources for network connectivity
  • Generate ConfigMap and Secret resources for configuration management
  • Create PersistentVolumeClaim manifests for stateful workloads
  • Follow Kubernetes best practices and naming conventions
  • Implement resource limits, health checks, and security contexts
  • Design manifests for multi-environment deployments

Detailed patterns and worked examples

Detailed pattern documentation lives in references/details.md. Read that file when the navigation tier above is insufficient.

Best Practices Summary

  1. Always set resource requests and limits - Prevents resource starvation
  2. Implement health checks - Ensures Kubernetes can manage your application
  3. Use specific image tags - Avoid unpredictable deployments
  4. Apply security contexts - Run as non-root, drop capabilities
  5. Use ConfigMaps and Secrets - Separate config from code
  6. Label everything - Enables filtering and organization
  7. Follow naming conventions - Use standard Kubernetes labels
  8. Validate before applying - Use dry-run and validation tools
  9. Version your manifests - Keep in Git with version control
  10. Document with annotations - Add context for other developers

Troubleshooting

Pods not starting:

  • Check image pull errors: kubectl describe pod <pod-name>
  • Verify resource availability: kubectl get nodes
  • Check events: kubectl get events --sort-by='.lastTimestamp'

Service not accessible:

  • Verify selector matches pod labels: kubectl get endpoints <service-name>
  • Check service type and port configuration
  • Test from within cluster: kubectl run debug --rm -it --image=busybox -- sh

ConfigMap/Secret not loading:

  • Verify names match in Deployment
  • Check namespace
  • Ensure resources exist: kubectl get configmap,secret

Next Steps

After creating manifests:

  1. Store in Git repository
  2. Set up CI/CD pipeline for deployment
  3. Consider using Helm or Kustomize for templating
  4. Implement GitOps with ArgoCD or Flux
  5. Add monitoring and observability

Related Skills

  • helm-chart-scaffolding - For templating and packaging
  • gitops-workflow - For automated deployments
  • k8s-security-policies - For advanced security configurations

Kubernetes Security Policies

Comprehensive guide for implementing NetworkPolicy, PodSecurityPolicy, RBAC, and Pod Security Standards in Kubernetes.

Purpose

Implement defense-in-depth security for Kubernetes clusters using network policies, pod security standards, and RBAC.

When to Use This Skill

  • Implement network segmentation
  • Configure pod security standards
  • Set up RBAC for least-privilege access
  • Create security policies for compliance
  • Implement admission control
  • Secure multi-tenant clusters

Pod Security Standards

1. Privileged (Unrestricted)

apiVersion: v1
kind: Namespace
metadata:
  name: privileged-ns
  labels:
    pod-security.kubernetes.io/enforce: privileged
    pod-security.kubernetes.io/audit: privileged
    pod-security.kubernetes.io/warn: privileged

2. Baseline (Minimally restrictive)

apiVersion: v1
kind: Namespace
metadata:
  name: baseline-ns
  labels:
    pod-security.kubernetes.io/enforce: baseline
    pod-security.kubernetes.io/audit: baseline
    pod-security.kubernetes.io/warn: baseline

3. Restricted (Most restrictive)

apiVersion: v1
kind: Namespace
metadata:
  name: restricted-ns
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/warn: restricted

Network Policies

Default Deny All

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: production
spec:
  podSelector: {}
  policyTypes:
    - Ingress
    - Egress

Allow Frontend to Backend

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-frontend-to-backend
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: backend
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: frontend
      ports:
        - protocol: TCP
          port: 8080

Allow DNS

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns
  namespace: production
spec:
  podSelector: {}
  policyTypes:
    - Egress
  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              name: kube-system
      ports:
        - protocol: UDP
          port: 53

Reference: See assets/network-policy-template.yaml

RBAC Configuration

Role (Namespace-scoped)

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: pod-reader
  namespace: production
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "watch", "list"]

ClusterRole (Cluster-wide)

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: secret-reader
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get", "watch", "list"]

RoleBinding

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-pods
  namespace: production
subjects:
  - kind: User
    name: jane
    apiGroup: rbac.authorization.k8s.io
  - kind: ServiceAccount
    name: default
    namespace: production
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

Reference: See references/rbac-patterns.md

Pod Security Context

Restricted Pod

apiVersion: v1
kind: Pod
metadata:
  name: secure-pod
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 1000
    fsGroup: 1000
    seccompProfile:
      type: RuntimeDefault
  containers:
    - name: app
      image: myapp:1.0
      securityContext:
        allowPrivilegeEscalation: false
        readOnlyRootFilesystem: true
        capabilities:
          drop:
            - ALL

Policy Enforcement with OPA Gatekeeper

ConstraintTemplate

apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: k8srequiredlabels
spec:
  crd:
    spec:
      names:
        kind: K8sRequiredLabels
      validation:
        openAPIV3Schema:
          type: object
          properties:
            labels:
              type: array
              items:
                type: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8srequiredlabels
        violation[{"msg": msg, "details": {"missing_labels": missing}}] {
          provided := {label | input.review.object.metadata.labels[label]}
          required := {label | label := input.parameters.labels[_]}
          missing := required - provided
          count(missing) > 0
          msg := sprintf("missing required labels: %v", [missing])
        }

Constraint

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
  name: require-app-label
spec:
  match:
    kinds:
      - apiGroups: ["apps"]
        kinds: ["Deployment"]
  parameters:
    labels: ["app", "environment"]

Service Mesh Security (Istio)

PeerAuthentication (mTLS)

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: production
spec:
  mtls:
    mode: STRICT

AuthorizationPolicy

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: allow-frontend
  namespace: production
spec:
  selector:
    matchLabels:
      app: backend
  action: ALLOW
  rules:
    - from:
        - source:
            principals: ["cluster.local/ns/production/sa/frontend"]

Best Practices

  1. Implement Pod Security Standards at namespace level
  2. Use Network Policies for network segmentation
  3. Apply least-privilege RBAC for all service accounts
  4. Enable admission control (OPA Gatekeeper/Kyverno)
  5. Run containers as non-root
  6. Use read-only root filesystem
  7. Drop all capabilities unless needed
  8. Implement resource quotas and limit ranges
  9. Enable audit logging for security events
  10. Regular security scanning of images

Compliance Frameworks

CIS Kubernetes Benchmark

  • Use RBAC authorization
  • Enable audit logging
  • Use Pod Security Standards
  • Configure network policies
  • Implement secrets encryption at rest
  • Enable node authentication

NIST Cybersecurity Framework

  • Implement defense in depth
  • Use network segmentation
  • Configure security monitoring
  • Implement access controls
  • Enable logging and monitoring

Troubleshooting

NetworkPolicy not working:

# Check if CNI supports NetworkPolicy
kubectl get nodes -o wide
kubectl describe networkpolicy <name>

RBAC permission denied:

# Check effective permissions
kubectl auth can-i list pods --as system:serviceaccount:default:my-sa
kubectl auth can-i '*' '*' --as system:serviceaccount:default:my-sa

Related Skills

  • k8s-manifest-generator - For creating secure manifests
  • gitops-workflow - For automated policy deployment

You are a Kubernetes architect specializing in cloud-native infrastructure, modern GitOps workflows, and enterprise container orchestration at scale.

Use this skill when

  • Designing Kubernetes platform architecture or multi-cluster strategy
  • Implementing GitOps workflows and progressive delivery
  • Planning service mesh, security, or multi-tenancy patterns
  • Improving reliability, cost, or developer experience in K8s

Do not use this skill when

  • You only need a local dev cluster or single-node setup
  • You are troubleshooting application code without platform changes
  • You are not using Kubernetes or container orchestration

Instructions

  1. Gather workload requirements, compliance needs, and scale targets.
  2. Define cluster topology, networking, and security boundaries.
  3. Choose GitOps tooling and delivery strategy for rollouts.
  4. Validate with staging and define rollback and upgrade plans.

Safety

  • Avoid production changes without approvals and rollback plans.
  • Test policy changes and admission controls in staging first.

Purpose

Expert Kubernetes architect with comprehensive knowledge of container orchestration, cloud-native technologies, and modern GitOps practices. Masters Kubernetes across all major providers (EKS, AKS, GKE) and on-premises deployments. Specializes in building scalable, secure, and cost-effective platform engineering solutions that enhance developer productivity.

Capabilities

Kubernetes Platform Expertise

  • Managed Kubernetes: EKS (AWS), AKS (Azure), GKE (Google Cloud), advanced configuration and optimization
  • Enterprise Kubernetes: Red Hat OpenShift, Rancher, VMware Tanzu, platform-specific features
  • Self-managed clusters: kubeadm, kops, kubespray, bare-metal installations, air-gapped deployments
  • Cluster lifecycle: Upgrades, node management, etcd operations, backup/restore strategies
  • Multi-cluster management: Cluster API, fleet management, cluster federation, cross-cluster networking

GitOps & Continuous Deployment

  • GitOps tools: ArgoCD, Flux v2, Jenkins X, Tekton, advanced configuration and best practices
  • OpenGitOps principles: Declarative, versioned, automatically pulled, continuously reconciled
  • Progressive delivery: Argo Rollouts, Flagger, canary deployments, blue/green strategies, A/B testing
  • GitOps repository patterns: App-of-apps, mono-repo vs multi-repo, environment promotion strategies
  • Secret management: External Secrets Operator, Sealed Secrets, HashiCorp Vault integration

Modern Infrastructure as Code

  • Kubernetes-native IaC: Helm 3.x, Kustomize, Jsonnet, cdk8s, Pulumi Kubernetes provider
  • Cluster provisioning: Terraform/OpenTofu modules, Cluster API, infrastructure automation
  • Configuration management: Advanced Helm patterns, Kustomize overlays, environment-specific configs
  • Policy as Code: Open Policy Agent (OPA), Gatekeeper, Kyverno, Falco rules, admission controllers
  • GitOps workflows: Automated testing, validation pipelines, drift detection and remediation

Cloud-Native Security

  • Pod Security Standards: Restricted, baseline, privileged policies, migration strategies
  • Network security: Network policies, service mesh security, micro-segmentation
  • Runtime security: Falco, Sysdig, Aqua Security, runtime threat detection
  • Image security: Container scanning, admission controllers, vulnerability management
  • Supply chain security: SLSA, Sigstore, image signing, SBOM generation
  • Compliance: CIS benchmarks, NIST frameworks, regulatory compliance automation

Service Mesh Architecture

  • Istio: Advanced traffic management, security policies, observability, multi-cluster mesh
  • Linkerd: Lightweight service mesh, automatic mTLS, traffic splitting
  • Cilium: eBPF-based networking, network policies, load balancing
  • Consul Connect: Service mesh with HashiCorp ecosystem integration
  • Gateway API: Next-generation ingress, traffic routing, protocol support

Container & Image Management

  • Container runtimes: containerd, CRI-O, Docker runtime considerations
  • Registry strategies: Harbor, ECR, ACR, GCR, multi-region replication
  • Image optimization: Multi-stage builds, distroless images, security scanning
  • Build strategies: BuildKit, Cloud Native Buildpacks, Tekton pipelines, Kaniko
  • Artifact management: OCI artifacts, Helm chart repositories, policy distribution

Observability & Monitoring

  • Metrics: Prometheus, VictoriaMetrics, Thanos for long-term storage
  • Logging: Fluentd, Fluent Bit, Loki, centralized logging strategies
  • Tracing: Jaeger, Zipkin, OpenTelemetry, distributed tracing patterns
  • Visualization: Grafana, custom dashboards, alerting strategies
  • APM integration: DataDog, New Relic, Dynatrace Kubernetes-specific monitoring

Multi-Tenancy & Platform Engineering

  • Namespace strategies: Multi-tenancy patterns, resource isolation, network segmentation
  • RBAC design: Advanced authorization, service accounts, cluster roles, namespace roles
  • Resource management: Resource quotas, limit ranges, priority classes, QoS classes
  • Developer platforms: Self-service provisioning, developer portals, abstract infrastructure complexity
  • Operator development: Custom Resource Definitions (CRDs), controller patterns, Operator SDK

Scalability & Performance

  • Cluster autoscaling: Horizontal Pod Autoscaler (HPA), Vertical Pod Autoscaler (VPA), Cluster Autoscaler
  • Custom metrics: KEDA for event-driven autoscaling, custom metrics APIs
  • Performance tuning: Node optimization, resource allocation, CPU/memory management
  • Load balancing: Ingress controllers, service mesh load balancing, external load balancers
  • Storage: Persistent volumes, storage classes, CSI drivers, data management

Cost Optimization & FinOps

  • Resource optimization: Right-sizing workloads, spot instances, reserved capacity
  • Cost monitoring: KubeCost, OpenCost, native cloud cost allocation
  • Bin packing: Node utilization optimization, workload density
  • Cluster efficiency: Resource requests/limits optimization, over-provisioning analysis
  • Multi-cloud cost: Cross-provider cost analysis, workload placement optimization

Disaster Recovery & Business Continuity

  • Backup strategies: Velero, cloud-native backup solutions, cross-region backups
  • Multi-region deployment: Active-active, active-passive, traffic routing
  • Chaos engineering: Chaos Monkey, Litmus, fault injection testing
  • Recovery procedures: RTO/RPO planning, automated failover, disaster recovery testing

OpenGitOps Principles (CNCF)

  1. Declarative - Entire system described declaratively with desired state
  2. Versioned and Immutable - Desired state stored in Git with complete version history
  3. Pulled Automatically - Software agents automatically pull desired state from Git
  4. Continuously Reconciled - Agents continuously observe and reconcile actual vs desired state

Behavioral Traits

  • Champions Kubernetes-first approaches while recognizing appropriate use cases
  • Implements GitOps from project inception, not as an afterthought
  • Prioritizes developer experience and platform usability
  • Emphasizes security by default with defense in depth strategies
  • Designs for multi-cluster and multi-region resilience
  • Advocates for progressive delivery and safe deployment practices
  • Focuses on cost optimization and resource efficiency
  • Promotes observability and monitoring as foundational capabilities
  • Values automation and Infrastructure as Code for all operations
  • Considers compliance and governance requirements in architecture decisions

Knowledge Base

  • Kubernetes architecture and component interactions
  • CNCF landscape and cloud-native technology ecosystem
  • GitOps patterns and best practices
  • Container security and supply chain best practices
  • Service mesh architectures and trade-offs
  • Platform engineering methodologies
  • Cloud provider Kubernetes services and integrations
  • Observability patterns and tools for containerized environments
  • Modern CI/CD practices and pipeline security

Response Approach

  1. Assess workload requirements for container orchestration needs
  2. Design Kubernetes architecture appropriate for scale and complexity
  3. Implement GitOps workflows with proper repository structure and automation
  4. Configure security policies with Pod Security Standards and network policies
  5. Set up observability stack with metrics, logs, and traces
  6. Plan for scalability with appropriate autoscaling and resource management
  7. Consider multi-tenancy requirements and namespace isolation
  8. Optimize for cost with right-sizing and efficient resource utilization
  9. Document platform with clear operational procedures and developer guides

Example Interactions

  • "Design a multi-cluster Kubernetes platform with GitOps for a financial services company"
  • "Implement progressive delivery with Argo Rollouts and service mesh traffic splitting"
  • "Create a secure multi-tenant Kubernetes platform with namespace isolation and RBAC"
  • "Design disaster recovery for stateful applications across multiple Kubernetes clusters"
  • "Optimize Kubernetes costs while maintaining performance and availability SLAs"
  • "Implement observability stack with Prometheus, Grafana, and OpenTelemetry for microservices"
  • "Create CI/CD pipeline with GitOps for container applications with security scanning"
  • "Design Kubernetes operator for custom application lifecycle management"

Limitations

  • Use this skill only when the task clearly matches the scope described above.
  • Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
  • Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.

Kubernetes Patterns

Production-grade Kubernetes patterns for deploying, managing, and debugging workloads reliably.

When to Activate

  • Writing Kubernetes manifests (Deployments, Services, Ingress, Jobs)
  • Configuring resource requests/limits, liveness/readiness probes
  • Setting up RBAC, namespaces, or ServiceAccounts
  • Managing configuration and secrets in K8s
  • Debugging CrashLoopBackOff, OOMKilled, pending pods, or image pull errors
  • Configuring HPA (Horizontal Pod Autoscaler) or PodDisruptionBudgets
  • Reviewing K8s YAML for security or correctness

When to Use

Same as When to Activate above. This alias satisfies repo skill-format conventions. Use this skill any time you are writing, reviewing, or debugging Kubernetes YAML and workloads.

How It Works

This skill provides copy-pasteable, production-grade YAML patterns and kubectl debugging commands organized by task:

  1. Deployment template — A fully configured production Deployment with security context, rolling update strategy, all three probe types, resource limits, and environment injection from ConfigMap/Secret.
  2. Probes — Decision table for startup vs liveness vs readiness, with correct failureThreshold × periodSeconds math.
  3. Services & Ingress — ClusterIP, LoadBalancer, and TLS Ingress patterns with cert-manager annotations.
  4. ConfigMaps & Secrets — envFrom, file-mount, and external secrets guidance.
  5. Resource management — Requests vs limits rules of thumb by workload type (web API, JVM, worker, sidecar).
  6. RBAC — Least-privilege ServiceAccount → Role → RoleBinding chain.
  7. HPA & PDB — Autoscaling and node-drain safety configurations.
  8. Jobs & CronJobs — One-off and scheduled workload patterns with correct restartPolicy.
  9. kubectl cheatsheet — Logs, exec, rollback, port-forward, dry-run, and common error diagnosis commands.
  10. Anti-patterns & checklist — What NOT to do, and a security/reliability/observability checklist.

Examples

See the sections below for complete, runnable examples. Quick references:

TaskJump to
Full production Deployment YAMLCore Workload Patterns
Probe configurationProbes
RBAC least-privilege setupRBAC
Debug a CrashLoopBackOffkubectl Debugging Cheatsheet
AutoscalingHPA

Core Workload Patterns

Deployment — Production Template

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  namespace: my-namespace
  labels:
    app: my-app
    version: "1.0.0"
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1          # Allow 1 extra pod during update
      maxUnavailable: 0    # Never reduce below desired count
  template:
    metadata:
      labels:
        app: my-app
        version: "1.0.0"
    spec:
      # Security context at pod level
      securityContext:
        runAsNonRoot: true
        runAsUser: 1001
        fsGroup: 1001

      # Graceful shutdown
      terminationGracePeriodSeconds: 30

      containers:
        - name: my-app
          image: ghcr.io/org/my-app:1.0.0   # Never use :latest
          imagePullPolicy: IfNotPresent

          ports:
            - containerPort: 8080
              protocol: TCP

          # Resource requests AND limits are both required
          resources:
            requests:
              cpu: "100m"
              memory: "128Mi"
            limits:
              cpu: "500m"
              memory: "256Mi"

          # Container security context
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop:
                - ALL

          # Probes (see Probes section below)
          startupProbe:
            httpGet:
              path: /health
              port: 8080
            failureThreshold: 30
            periodSeconds: 5
          livenessProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 0
            periodSeconds: 30
            failureThreshold: 3
          readinessProbe:
            httpGet:
              path: /ready
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10
            failureThreshold: 2

          # Environment from ConfigMap and Secret
          envFrom:
            - configMapRef:
                name: my-app-config
          env:
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: my-app-secrets
                  key: db-password

          # Writable tmp directory when readOnlyRootFilesystem: true
          volumeMounts:
            - name: tmp
              mountPath: /tmp

      volumes:
        - name: tmp
          emptyDir: {}

Probes — Liveness, Readiness, Startup

Understanding when to use each probe is critical:

ProbeFailure ActionUse For
startupProbeKills container if slow to startSlow-starting apps (JVM, Python)
livenessProbeRestarts containerDeadlock / hung process detection
readinessProbeRemoves from Service endpointsTemporary unavailability (DB reconnect)
# Correct pattern: startupProbe covers slow startup,
# then liveness/readiness take over
startupProbe:
  httpGet:
    path: /health
    port: 8080
  failureThreshold: 30  # 30 * 5s = 150s max startup time
  periodSeconds: 5

livenessProbe:
  httpGet:
    path: /health
    port: 8080
  periodSeconds: 30
  failureThreshold: 3   # 3 * 30s = 90s before restart

readinessProbe:
  httpGet:
    path: /ready         # Separate endpoint: checks DB, cache, etc.
    port: 8080
  periodSeconds: 10
  failureThreshold: 2
# WRONG: initialDelaySeconds without startupProbe
# If the app takes 60s to start, set a startupProbe instead
livenessProbe:
  httpGet:
    path: /health
    port: 8080
  initialDelaySeconds: 60   # BAD: Arbitrary wait, race condition

Services and Ingress

Service Types

# ClusterIP (default) — internal-only
apiVersion: v1
kind: Service
metadata:
  name: my-app
  namespace: my-namespace
spec:
  selector:
    app: my-app
  ports:
    - port: 80
      targetPort: 8080
      protocol: TCP
  type: ClusterIP
# LoadBalancer — external traffic (cloud providers)
spec:
  type: LoadBalancer
  ports:
    - port: 443
      targetPort: 8080

Ingress with TLS

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-app
  namespace: my-namespace
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - myapp.example.com
      secretName: my-app-tls
  rules:
    - host: myapp.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-app
                port:
                  number: 80

ConfigMaps and Secrets

ConfigMap — Non-sensitive configuration

apiVersion: v1
kind: ConfigMap
metadata:
  name: my-app-config
  namespace: my-namespace
data:
  LOG_LEVEL: "info"
  APP_ENV: "production"
  MAX_CONNECTIONS: "100"
  # Mount as a file for complex config
  app.yaml: |
    server:
      port: 8080
      timeout: 30s
# Mount ConfigMap as a file
volumes:
  - name: config
    configMap:
      name: my-app-config
      items:
        - key: app.yaml
          path: app.yaml
volumeMounts:
  - name: config
    mountPath: /etc/app
    readOnly: true

Secrets — Sensitive data

# Create secret from literal (CLI, then store in Vault/SOPS)
kubectl create secret generic my-app-secrets \
  --from-literal=db-password='s3cr3t' \
  --namespace=my-namespace \
  --dry-run=client -o yaml | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: my-app-secrets
  namespace: my-namespace
type: Opaque
# Values are base64-encoded (NOT encrypted — use Sealed Secrets or ESO for real encryption)
data:
  db-password: czNjcjN0  # base64 of 's3cr3t'

Important: Raw Kubernetes Secrets are only base64-encoded, not encrypted at rest unless your cluster has encryption configured. Use Sealed Secrets or External Secrets Operator for production.


Resource Requests and Limits

resources:
  requests:       # Scheduler uses this to place the pod
    cpu: "100m"   # 100 millicores = 0.1 CPU
    memory: "128Mi"
  limits:         # Container is killed/throttled above this
    cpu: "500m"
    memory: "256Mi"

Rules of thumb:

Workload TypeCPU RequestMemory RequestNotes
Web API100–250m128–256MiSet limits 2-4x requests
Worker/consumer250–500m256–512MiMemory limit = request for predictability
JVM app500m–1512Mi–2GiAllow headroom above -Xmx for JVM overhead
Sidecar10–50m32–64MiKeep minimal
# WRONG: No requests or limits — unpredictable scheduling, OOM evictions
containers:
  - name: app
    image: myapp:latest
    # Missing resources: {} — this is dangerous in production

# WRONG: Limits without requests — requests default to limits, over-reserves capacity
resources:
  limits:
    cpu: "2"
    memory: "1Gi"
  # requests missing — will default to limits values

RBAC — Roles and ServiceAccounts

Principle of Least Privilege

Two patterns depending on whether the app calls the Kubernetes API:

Pattern A — App does NOT need the Kubernetes API (most apps)

Disable token automounting on the ServiceAccount. The Role/RoleBinding are not needed.

# ServiceAccount with token disabled — safest default
apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-app-sa
  namespace: my-namespace
automountServiceAccountToken: false   # No K8s API token injected into pods
# Reference in Deployment — no token, no API access
spec:
  template:
    spec:
      serviceAccountName: my-app-sa
      automountServiceAccountToken: false   # Belt-and-suspenders: also set at pod level
Pattern B — App DOES need the Kubernetes API (operators, controllers, config watchers)

Enable the token and grant only the permissions actually required.

# 1. ServiceAccount — enable token for this SA
apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-app-sa
  namespace: my-namespace
automountServiceAccountToken: true    # Token required: app calls K8s API
# 2. Role — grant only what the app needs (namespace-scoped)
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: my-app-role
  namespace: my-namespace
rules:
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["get", "list", "watch"]    # Read-only, specific resource
  - apiGroups: [""]
    resources: ["secrets"]
    resourceNames: ["my-app-secrets"]  # Restrict to specific secret by name
    verbs: ["get"]
# 3. Bind Role to ServiceAccount
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: my-app-rolebinding
  namespace: my-namespace
subjects:
  - kind: ServiceAccount
    name: my-app-sa
    namespace: my-namespace
roleRef:
  kind: Role
  apiGroup: rbac.authorization.k8s.io
  name: my-app-role
# 4. Reference SA in Deployment
spec:
  template:
    spec:
      serviceAccountName: my-app-sa
      # automountServiceAccountToken defaults to true from SA — token is injected

Horizontal Pod Autoscaler (HPA)

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: my-app-hpa
  namespace: my-namespace
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  minReplicas: 2      # Always at least 2 for HA
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70    # Scale up when avg CPU > 70%
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80

HPA requires resources.requests to be set on all containers — it calculates utilization as current / request.


PodDisruptionBudget (PDB)

Prevent too many pods going down during node drains or rolling updates:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: my-app-pdb
  namespace: my-namespace
spec:
  minAvailable: 2           # OR use maxUnavailable: 1
  selector:
    matchLabels:
      app: my-app

Namespaces and Multi-Tenancy

# Create namespace with resource quotas
kubectl create namespace my-namespace

# Apply ResourceQuota to limit namespace consumption
kubectl apply -f - <<EOF
apiVersion: v1
kind: ResourceQuota
metadata:
  name: my-namespace-quota
  namespace: my-namespace
spec:
  hard:
    requests.cpu: "4"
    requests.memory: 4Gi
    limits.cpu: "8"
    limits.memory: 8Gi
    pods: "20"
EOF

Jobs and CronJobs

# One-off Job (DB migration, data processing)
apiVersion: batch/v1
kind: Job
metadata:
  name: db-migrate
  namespace: my-namespace
spec:
  backoffLimit: 3          # Retry up to 3 times on failure
  ttlSecondsAfterFinished: 3600   # Auto-delete after 1h
  template:
    spec:
      restartPolicy: OnFailure    # Never for Jobs (not Always)
      containers:
        - name: migrate
          image: ghcr.io/org/my-app:1.0.0
          command: ["python", "manage.py", "migrate"]
          resources:
            requests:
              cpu: "100m"
              memory: "256Mi"
# CronJob
apiVersion: batch/v1
kind: CronJob
metadata:
  name: cleanup-job
  namespace: my-namespace
spec:
  schedule: "0 2 * * *"         # 2am daily
  concurrencyPolicy: Forbid      # Don't run if previous still running
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 1
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: OnFailure
          containers:
            - name: cleanup
              image: ghcr.io/org/cleanup:1.0.0
              resources:
                requests:
                  cpu: "50m"
                  memory: "64Mi"

kubectl Debugging Cheatsheet

# --- Pod status and logs ---
kubectl get pods -n my-namespace
kubectl get pods -n my-namespace -o wide          # Show node assignment
kubectl describe pod <pod-name> -n my-namespace   # Events and state details
kubectl logs <pod-name> -n my-namespace           # Current logs
kubectl logs <pod-name> -n my-namespace --previous  # Logs from crashed container
kubectl logs <pod-name> -n my-namespace -c <container>  # Multi-container pod

# --- Execute into a running container ---
kubectl exec -it <pod-name> -n my-namespace -- sh
kubectl exec -it <pod-name> -n my-namespace -- bash

# --- Check resource usage ---
kubectl top pods -n my-namespace
kubectl top nodes

# --- Deployment operations ---
kubectl rollout status deployment/my-app -n my-namespace
kubectl rollout history deployment/my-app -n my-namespace
kubectl rollout undo deployment/my-app -n my-namespace      # Rollback
kubectl rollout undo deployment/my-app --to-revision=2 -n my-namespace

# --- Scale manually ---
kubectl scale deployment my-app --replicas=5 -n my-namespace

# --- Inspect events (cluster-wide issues) ---
kubectl get events -n my-namespace --sort-by='.lastTimestamp'

# --- Port-forward for local debugging ---
kubectl port-forward pod/<pod-name> 8080:8080 -n my-namespace
kubectl port-forward svc/my-app 8080:80 -n my-namespace

# --- Dry-run to validate YAML ---
kubectl apply -f deployment.yaml --dry-run=client
kubectl apply -f deployment.yaml --dry-run=server   # Validates against live cluster

Diagnosing Common Errors

# CrashLoopBackOff: container keeps crashing
kubectl logs <pod-name> --previous -n my-namespace  # Check crash logs
kubectl describe pod <pod-name> -n my-namespace     # Check exit code & OOMKilled

# ImagePullBackOff: can't pull image
kubectl describe pod <pod-name> -n my-namespace     # Check Events section
# Causes: wrong image tag, missing imagePullSecret, private registry

# Pending pod: not scheduled
kubectl describe pod <pod-name> -n my-namespace
# Causes: insufficient resources, no matching node selector, taint/toleration mismatch

# OOMKilled: out of memory
# Increase memory limits, check for memory leaks
kubectl describe pod <pod-name> -n my-namespace | grep -A5 "Last State"

Anti-Patterns

# BAD: Using :latest tag — non-deterministic deployments
image: myapp:latest

# GOOD: Pin to a specific immutable tag (SHA or semver)
image: ghcr.io/org/myapp:1.4.2
# or
image: ghcr.io/org/myapp@sha256:abc123...

# ---

# BAD: Running as root
securityContext: {}    # Defaults to root

# GOOD: Non-root with explicit UID
securityContext:
  runAsNonRoot: true
  runAsUser: 1001

# ---

# BAD: No resource limits — one pod can starve the entire node
containers:
  - name: app
    image: myapp:1.0.0
    # No resources defined

# GOOD: Always set requests and limits
resources:
  requests:
    cpu: "100m"
    memory: "128Mi"
  limits:
    cpu: "500m"
    memory: "256Mi"

# ---

# BAD: Storing plaintext secrets in ConfigMaps
apiVersion: v1
kind: ConfigMap
data:
  DB_PASSWORD: "mysecretpassword"   # NEVER — use Secret or external secrets manager

# ---

# BAD: ClusterAdmin for application service accounts
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
roleRef:
  kind: ClusterRole
  name: cluster-admin    # Grants god-mode to your app

# ---

# BAD: minAvailable: 0 in PDB — defeats the purpose
spec:
  minAvailable: 0

# ---

# BAD: restartPolicy: Always in a Job (causes infinite restart loop)
spec:
  restartPolicy: Always   # Use OnFailure or Never for Jobs

Best Practices Checklist

Security

  • Container runs as non-root (runAsNonRoot: true, runAsUser set)
  • readOnlyRootFilesystem: true with emptyDir for writable paths
  • allowPrivilegeEscalation: false
  • All capabilities dropped (capabilities.drop: [ALL])
  • Dedicated ServiceAccount per app, not default
  • automountServiceAccountToken: false unless needed
  • RBAC follows least privilege (use Role, not ClusterRole unless needed)
  • Secrets managed via Sealed Secrets or External Secrets Operator

Reliability

  • All 3 probe types configured (startup + liveness + readiness)
  • Resource requests AND limits set on every container
  • minReplicas: 2+ for any production workload
  • PodDisruptionBudget defined for stateful or critical services
  • RollingUpdate strategy with maxUnavailable: 0
  • HPA configured for variable-load services

Observability

  • App exposes /health (liveness) and /ready (readiness) endpoints
  • Structured JSON logging (no PII in logs)
  • Resource labels: app, version, environment

Related Skills

  • docker-patterns — Multi-stage Dockerfiles and image security
  • deployment-patterns — CI/CD pipelines, rollback strategy, health check endpoints
  • security-review — Broader security hardening context
  • git-workflow — GitOps integration with K8s (ArgoCD / Flux patterns)

Linkerd Patterns

Production patterns for Linkerd service mesh - the lightweight, security-first service mesh for Kubernetes.

When to Use This Skill

  • Setting up a lightweight service mesh
  • Implementing automatic mTLS
  • Configuring traffic splits for canary deployments
  • Setting up service profiles for per-route metrics
  • Implementing retries and timeouts
  • Multi-cluster service mesh

Core Concepts

1. Linkerd Architecture

┌─────────────────────────────────────────────┐
│                Control Plane                 │
│  ┌─────────┐ ┌──────────┐ ┌──────────────┐ │
│  │ destiny │ │ identity │ │ proxy-inject │ │
│  └─────────┘ └──────────┘ └──────────────┘ │
└─────────────────────────────────────────────┘
                      │
┌─────────────────────────────────────────────┐
│                 Data Plane                   │
│  ┌─────┐    ┌─────┐    ┌─────┐             │
│  │proxy│────│proxy│────│proxy│             │
│  └─────┘    └─────┘    └─────┘             │
│     │           │           │               │
│  ┌──┴──┐    ┌──┴──┐    ┌──┴──┐            │
│  │ app │    │ app │    │ app │            │
│  └─────┘    └─────┘    └─────┘            │
└─────────────────────────────────────────────┘

2. Key Resources

ResourcePurpose
ServiceProfilePer-route metrics, retries, timeouts
TrafficSplitCanary deployments, A/B testing
ServerDefine server-side policies
ServerAuthorizationAccess control policies

Templates

Template 1: Mesh Installation

Use a CLI version compatible with the target cluster. The example uses Homebrew; for other systems, follow the official installation guide.

# Install the CLI with Homebrew
brew install linkerd
linkerd version

# Validate cluster
linkerd check --pre

# Install CRDs
linkerd install --crds | kubectl apply -f -

# Install control plane
linkerd install | kubectl apply -f -

# Verify installation
linkerd check

# Install viz extension (optional)
linkerd viz install | kubectl apply -f -

Template 2: Inject Namespace

# Automatic injection for namespace
apiVersion: v1
kind: Namespace
metadata:
  name: my-app
  annotations:
    linkerd.io/inject: enabled
---
# Or inject specific deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  annotations:
    linkerd.io/inject: enabled
spec:
  template:
    metadata:
      annotations:
        linkerd.io/inject: enabled

Template 3: Service Profile with Retries

apiVersion: linkerd.io/v1alpha2
kind: ServiceProfile
metadata:
  name: my-service.my-namespace.svc.cluster.local
  namespace: my-namespace
spec:
  routes:
    - name: GET /api/users
      condition:
        method: GET
        pathRegex: /api/users
      responseClasses:
        - condition:
            status:
              min: 500
              max: 599
          isFailure: true
      isRetryable: true
    - name: POST /api/users
      condition:
        method: POST
        pathRegex: /api/users
      # POST not retryable by default
      isRetryable: false
    - name: GET /api/users/{id}
      condition:
        method: GET
        pathRegex: /api/users/[^/]+
      timeout: 5s
      isRetryable: true
  retryBudget:
    retryRatio: 0.2
    minRetriesPerSecond: 10
    ttl: 10s

Template 4: Traffic Split (Canary)

apiVersion: split.smi-spec.io/v1alpha1
kind: TrafficSplit
metadata:
  name: my-service-canary
  namespace: my-namespace
spec:
  service: my-service
  backends:
    - service: my-service-stable
      weight: 900m # 90%
    - service: my-service-canary
      weight: 100m # 10%

Template 5: Server Authorization Policy

# Define the server
apiVersion: policy.linkerd.io/v1beta1
kind: Server
metadata:
  name: my-service-http
  namespace: my-namespace
spec:
  podSelector:
    matchLabels:
      app: my-service
  port: http
  proxyProtocol: HTTP/1
---
# Allow traffic from specific clients
apiVersion: policy.linkerd.io/v1beta1
kind: ServerAuthorization
metadata:
  name: allow-frontend
  namespace: my-namespace
spec:
  server:
    name: my-service-http
  client:
    meshTLS:
      serviceAccounts:
        - name: frontend
          namespace: my-namespace
---
# Allow unauthenticated traffic (e.g., from ingress)
apiVersion: policy.linkerd.io/v1beta1
kind: ServerAuthorization
metadata:
  name: allow-ingress
  namespace: my-namespace
spec:
  server:
    name: my-service-http
  client:
    unauthenticated: true
    networks:
      - cidr: 10.0.0.0/8

Template 6: HTTPRoute for Advanced Routing

apiVersion: policy.linkerd.io/v1beta2
kind: HTTPRoute
metadata:
  name: my-route
  namespace: my-namespace
spec:
  parentRefs:
    - name: my-service
      kind: Service
      group: core
      port: 8080
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /api/v2
        - headers:
            - name: x-api-version
              value: v2
      backendRefs:
        - name: my-service-v2
          port: 8080
    - matches:
        - path:
            type: PathPrefix
            value: /api
      backendRefs:
        - name: my-service-v1
          port: 8080

Template 7: Multi-cluster Setup

# On each cluster, install with cluster credentials
linkerd multicluster install | kubectl apply -f -

# Link clusters
linkerd multicluster link --cluster-name west \
  --api-server-address https://west.example.com:6443 \
  | kubectl apply -f -

# Export a service to other clusters
kubectl label svc/my-service mirror.linkerd.io/exported=true

# Verify cross-cluster connectivity
linkerd multicluster check
linkerd multicluster gateways

Monitoring Commands

# Live traffic view
linkerd viz top deploy/my-app

# Per-route metrics
linkerd viz routes deploy/my-app

# Check proxy status
linkerd viz stat deploy -n my-namespace

# View service dependencies
linkerd viz edges deploy -n my-namespace

# Dashboard
linkerd viz dashboard

Debugging

# Check injection status
linkerd check --proxy -n my-namespace

# View proxy logs
kubectl logs deploy/my-app -c linkerd-proxy

# Debug identity/TLS
linkerd identity -n my-namespace

# Tap traffic (live)
linkerd viz tap deploy/my-app --to deploy/my-backend

Best Practices

Do's

  • Enable mTLS everywhere - It's automatic with Linkerd
  • Use ServiceProfiles - Get per-route metrics and retries
  • Set retry budgets - Prevent retry storms
  • Monitor golden metrics - Success rate, latency, throughput

Don'ts

  • Don't skip check - Always run linkerd check after changes
  • Don't over-configure - Linkerd defaults are sensible
  • Don't ignore ServiceProfiles - They unlock advanced features
  • Don't forget timeouts - Set appropriate values per route

Uncloud Cluster Management

Reference for the uc CLI — a decentralised self-hosting platform using Docker containers, WireGuard mesh networking, and Caddy reverse proxy.

When to Activate

Use this skill when working with Uncloud clusters, especially when:

  • Bootstrapping or joining machines with uc machine
  • Deploying services from Compose files with uc deploy
  • Publishing HTTP, HTTPS, TCP, or UDP ports through Uncloud
  • Configuring Caddy ingress with x-caddy, x-ports, or --caddyfile
  • Routing external LAN devices through the cluster proxy
  • Inspecting logs, service state, volumes, DNS, or machine placement

How It Works

Uncloud runs Docker services across peer machines connected by a WireGuard mesh. Each machine is an equal cluster member; services communicate on the overlay network and Caddy runs globally to terminate public HTTP/HTTPS traffic. Compose files can use Uncloud extensions for ingress, placement, and generated Caddy configuration, while the uc CLI handles image distribution, scheduling, scaling, logs, and cluster state.

Examples

uc machine init user@host --name machine-1
uc service run --name web -p app.example.com:8080/https nginx:latest
uc deploy

Core Concepts

  • No central control plane — all machines are equal peers connected by WireGuard
  • Caddy runs as a global service on every machine; auto-obtains TLS from Let's Encrypt
  • Overlay network — services communicate via 10.210.0.0/16 by default; DNS provided inside the mesh
  • Caddyfile is autogenerated — never edit it directly; use x-caddy / --caddyfile instead

CLI Quick Reference

Machines

CommandPurpose
uc machine init user@hostBootstrap first machine / new cluster
uc machine add user@hostJoin machine to existing cluster
uc machine lsList machines
uc machine update NAME --public-ip IPUpdate public IP for ingress
uc machine rm NAMERemove machine

Key init flags: --name, --network 10.210.0.0/16, --no-caddy, --no-dns, --public-ip auto\|IP\|none

Services

CommandPurpose
uc service ls / uc lsList services
uc service run IMAGERun a single container service
uc deployDeploy from compose.yaml
uc deploy --no-buildDeploy already-pushed images without rebuilding
uc deploy --recreateForce service recreation
uc scale SERVICE NSet replica count
uc service logs SERVICEView logs
uc service exec SERVICEShell into container
uc service inspect SERVICEDetailed info
uc service rm SERVICERemove service (keeps named volumes)
uc psAll containers across cluster

Images

uc image push myapp:latest                    # Push local image to all machines
uc image push myapp:latest -m machine1,machine2  # Push to specific machines
uc images                                     # List images in cluster

Volumes

uc volume ls                  # All volumes
uc volume ls -m machine1      # On specific machine
uc volume create NAME -m MACHINE
uc volume rm NAME

Caddy

uc caddy config    # Show current generated Caddyfile (read-only)
uc caddy deploy    # Deploy/upgrade Caddy across cluster

DNS & Context

uc dns show        # Show reserved *.uncld.dev domain
uc dns reserve     # Reserve a new domain
uc ctx ls          # List cluster contexts
uc ctx use prod    # Switch context

Port Publishing

HTTP/HTTPS (via Caddy reverse proxy)

-p [hostname:]container_port[/protocol]
ExampleMeaning
-p 8080/httpsHTTPS with auto service-name.cluster-domain hostname
-p app.example.com:8080/httpsHTTPS with custom hostname
-p 8080/httpHTTP only, no TLS

TCP/UDP (host-bound, bypasses Caddy)

-p [host_ip:]host_port:container_port[/protocol]@host
ExampleMeaning
-p 5432:5432@hostTCP 5432 on all interfaces
-p 127.0.0.1:5432:5432@hostTCP 5432 loopback only
-p 53:5353/udp@hostUDP

Compose File Extensions

Uncloud adds these extensions on top of Docker Compose:

x-ports — publish ports with domains

services:
  app:
    image: app:latest
    x-ports:
      - example.com:8000/https
      - www.example.com:8000/https
      - api.example.com:9000/https

x-caddy — custom Caddy config for service

services:
  app:
    image: app:latest
    x-caddy: |
      example.com {
        redir https://www.example.com{uri} permanent
      }
      www.example.com {
        reverse_proxy {{upstreams 8000}} {
          import common_proxy
        }
        basic_auth /admin/* {
          admin $2a$14$...
        }
      }

Template functions available inside x-caddy:

  • {{upstreams [service] [port]}} — healthy container IPs
  • {{.Name}} — service name
  • {{.Upstreams}} — map of all services → IPs

x-machines — placement constraints

services:
  db:
    image: postgres:18
    x-machines: db-machine          # Single machine name
  app:
    image: app:latest
    x-machines:
      - machine-1
      - machine-2

Full multi-service example

services:
  api:
    build: ./api
    x-ports:
      - api.example.com:3000/https
    environment:
      DATABASE_URL: postgres://db:5432/mydb

  web:
    build: ./web
    x-ports:
      - example.com:8000/https
      - www.example.com:8000/https
    environment:
      API_URL: http://api:3000

  db:
    image: postgres:18
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
    volumes:
      - db-data:/var/lib/postgresql/data
    x-machines: db-machine

volumes:
  db-data:

Routing to External (Non-Cluster) Devices

To expose an external device (e.g. BMC, NAS, router UI) via Caddy without running a real container:

1. Create a Caddyfile snippet (e.g. ~/device.caddyfile):

https://device.example.com {
    reverse_proxy https://192.168.1.x {
        transport http {
            tls_insecure_skip_verify   # needed for self-signed BMC certs
        }
    }
    log
}

For plaintext upstream: reverse_proxy http://192.168.1.x:port

2. Register as a named service with no-op container:

uc service run \
  --name device-bmc \
  --caddyfile ~/device.caddyfile \
  registry.k8s.io/pause:3.9

pause is a minimal no-op container — it does nothing, but gives Uncloud a service entry to attach the Caddyfile to.

3. Verify:

uc caddy config   # device.example.com block should appear

--caddyfile cannot be combined with non-@host published ports.

DNS tip: A wildcard record (*.yourdomain.com → cluster-public-ip) means any new subdomain works immediately — no DNS change needed per service.


Service DNS (Internal)

Services inside the cluster resolve each other by name:

DNS nameResolves to
service-nameAny healthy container
service-name.internalSame
rr.service-name.internalRound-robin
nearest.service-name.internalMachine-local first

Scaling & Global Services

uc scale web 5    # 5 replicas (spread across machines)
uc scale web 1    # Scale down
services:
  caddy:
    deploy:
      mode: global   # One container on every machine

Image Tag Templates (in compose.yaml)

image: myapp:{{gitdate "20060102"}}.{{gitsha 7}}
image: myapp:{{gitsha 7}}.${GITHUB_RUN_ID:-local}
FunctionOutput
{{gitsha N}}First N chars of commit SHA
{{gitdate "format"}}Git commit date in Go format
{{date "format"}}Current date

Common Workflows

Deploy from source:

uc deploy                          # Build + push + deploy
uc build --push && uc deploy --no-build   # Separate steps

Inspect a service:

uc inspect web
uc logs -f web
uc logs --since 1h web
uc exec web                        # Opens shell
uc exec web /bin/sh -c "env"       # Run specific command

Zero-downtime deploys happen automatically; Uncloud waits for health checks before terminating old containers.

Force recreate:

uc deploy --recreate

Common Mistakes

MistakeFix
Editing the Caddyfile directlyUse x-caddy in compose or --caddyfile on uc service run
Proxying an HTTPS upstream with self-signed certAdd transport http { tls_insecure_skip_verify }
uc caddy config shows no user-defined blocksCaddy admin socket unreachable — check uc inspect caddy and uc logs caddy
Service can't reach external LAN IP from containerVerify Caddy container's host can route to target network
Volumes lost after uc service rmNamed volumes persist; only anonymous volumes are auto-removed

Sponsors

Details

Marketplace
marslanalvi/SkillsStack
Source
plugins/devops/containers-kubernetes
Category
Devops
Keywords
devops, containers-kubernetes

More from marslanalvi/SkillsStack

All 45