Docker instancer

The Docker instancer runs per-team challenge services on a standalone Docker host. It creates the containers, networks, volumes, expiration records, and Traefik routes needed for each instance. The bundled Compose deployment includes everything required to run it.

For the participant controls, common instancerConfig fields, and available endpoint types, see Instancer.

Deployment

The deployment files are in deploy/docker-instancer/.

The Compose stack exposes Traefik on 80, 443, and 1337. The instancer API itself binds to 127.0.0.1:12237 on the host and to tiny-instancer:1337 inside the rctf_network Docker network.

1
ports:
2
  - '12237:1337'

The environment file requires shared auth, Redis, and public instance host settings:

DEV_ENV=false
BIND_PORT=1337
WEB_WORKERS=1
USE_PROXY_HEADERS=true

AUTH_TOKEN=<shared-secret>
INSTANCES_HOST=instancer.example.com

REDIS_HOST=cache
REDIS_PORT_NUMBER=6379
REDIS_PASSWORD=<redis-password>

TRAEFIK_PERMANENT_REDIRECT_MIDDLEWARE_NAME=permanent-https-redirect@file

The Docker instancer stack starts from the repository root:

docker compose -f deploy/docker-instancer/compose.yml up -d

The same token belongs in rCTF:

1
instancers:
2
  docker:
3
    name: instancers/docker
4
    options:
5
      apiUrl: http://tiny-instancer:1337
6
      authToken: <shared-secret>

When rCTF is outside the rctf_network Docker network, the host-mapped API URL is:

1
instancers:
2
  docker:
3
    name: instancers/docker
4
    options:
5
      apiUrl: http://127.0.0.1:12237
6
      authToken: <shared-secret>

Traefik terminates TLS for every https and tcp-ssl endpoint. Each instance is served at <hostPrefix>-<uid>.<INSTANCES_HOST>, so a single wildcard certificate for *.<INSTANCES_HOST> covers every per-team instance. The static config (conf/tls.yml) points Traefik at two files in the mounted certs/ directory:

Instance hostnames are one label deep, so a single-level wildcard covers every <hostPrefix>. Generate the certificate with an ACME client that supports DNS-01, since wildcard certificates cannot use HTTP-01. With Certbot:

certbot certonly --manual --preferred-challenges dns -d '*.instancer.example.com'

Copy the issued files into the mount, matching the names Traefik expects:

cp /etc/letsencrypt/live/instancer.example.com/fullchain.pem deploy/docker-instancer/certs/fullchain.pem
cp /etc/letsencrypt/live/instancer.example.com/privkey.pem deploy/docker-instancer/certs/privkey.pem

Traefik loads these at startup and doesn’t watch the file config (watch: false), so restart the Traefik container after renewing or replacing the certificate.

Docker daemon address pool

Every instanced challenge creates its own Docker network, so the daemon’s default address pool can run out after roughly 30 networks. When that happens, docker network create fails with could not find an available, non-overlapping IPv4 address pool.

Expand the pool by adding the following to /etc/docker/daemon.json on the instancer host:

1
{
2
  "default-address-pools": [
3
    {
4
      "base": "100.64.0.0/10",
5
      "size": 24
6
    }
7
  ]
8
}

This splits the CGNAT range into 16,384 /24 networks. Add more pools if your deployment needs additional space.

Restart Docker after editing the file with systemctl restart docker. Existing networks keep their original ranges.

Docker challenge config

The Docker provider accepts a Docker Compose-like object under config:

1
instancerConfig:
2
  challengeIntegrationId: web-demo
3
  timeoutMilliseconds: 600000
4
  extendable: true
5
  config:
6
    services:
7
      app:
8
        image: ghcr.io/example/web-demo:latest
9
        environment:
10
          FLAG: flag{example}
11
        networks:
12
          - internal
13
        expose:
14
          - '8080'
15
        read_only: true
16
        mem_limit: 128m
17
        cpus: 0.5
18
        pids_limit: 128
19
    networks:
20
      internal:
21
        internal: true
22
  expose:
23
    - kind: https
24
      hostPrefix: web-demo
25
      containerName: app
26
      containerPort: 8080
27
      title: Challenge

The top-level Docker config supports services, networks, and volumes. At least one service is required after defaults are applied.

Each service supports these field groups:

Explicit services default to a read-only root filesystem, no-new-privileges, dropped Linux capabilities, 6m memory, 1.0 CPU, 1024 PIDs, and nofile soft and hard limits of 1024.

Services may only reference networks declared under config.networks. Named volume mounts may only reference volumes declared under config.volumes. Absolute and relative bind-style mount sources aren’t checked against the volume map.

Network isolation

Services do not join Docker’s shared default bridge unless configured to do so. With no networks, expose, or network_mode, a service gets network_mode: none.

When several challenge services need to communicate, declare a user-defined network with internal: true and attach each service to it. They can then reach one another without reaching the host or internet. Use network_mode: bridge only when the service specifically needs Docker’s default bridge.