Kubernetes instancer

The Kubernetes instancer runs each team’s challenge instance in its own namespace. A controller inside the cluster watches ChallengeInstance resources and creates the pods, network rules, services, and Traefik routes they describe.

rCTF communicates with the Kubernetes API through the instancers/k8s provider. The controller handles the resources inside the cluster.

How requests become instances

A deployment has three main components:

A participant request flows through these in order:

Namespaces use the predictable name inst-<challenge-id>-<team-id>. Child resources point back to their ChallengeInstance, allowing Kubernetes garbage collection to remove anything left behind after cleanup.

Prerequisites

The Terraform example assumes GKE plus Cloudflare for DNS and ACME. GCP Cloud DNS works as a drop-in alternative.

The instancer’s public hostname is <instancer_subdomain>.<instancer_zone>, or <instancer_zone> when no subdomain is configured. Each instance gets a hostname one level below it.

Controller image

The operator image is published at ghcr.io/otter-sec/rctf/k8s-operator. The Terraform module installs it from apps/k8s-operator/dist/install.yaml and fills in the configured INSTANCER_HOST, so you do not need to build or publish an image before running terraform apply.

Terraform variables

Start with the Terraform example in deploy/terraform/instancer/example/.

Copy terraform.tfvars.example to terraform.tfvars and fill in:

A minimal Cloudflare-backed file looks like this:

1
cloudflare_api_token = "<cloudflare-api-token>"
2
letsencrypt_email_address = "ops@example.com"
3
instancer_zone = "ctf.example.com"
4
instancer_subdomain = "instances"
5
ctf_name = "Example CTF"
6

7
gcp_project_id = "example-ctf"
8
gcp_region = "us-central1"
9
gcp_zone = "us-central1-a"
10
gcp_instancer_cluster_name = "rctf-cluster"
11
gcp_instancer_machine_type = "e2-standard-4"
12
gcp_instancer_min_node_count = 1
13
gcp_instancer_max_node_count = 8

To use GCP Cloud DNS instead of Cloudflare, comment out the Cloudflare blocks in dns.tf and tls.tf, uncomment the google_dns_record_set and gcloud ACME blocks, and set gcp_dns_managed_zone_name.

Deployment

What Terraform provisions

The example layers the GKE module, the k8s module, and the example-level resources in rctf-operator.tf, dns.tf, and tls.tf:

Traefik is configured with three ports:

Terraform requests the wildcard certificate outside the cluster, so DNS provider credentials are never stored there. A compromised cluster can expose the issued certificate, but it cannot use those credentials to request new ones or change DNS records.

Network policies

The controller creates three NetworkPolicy resources in every instance namespace:

The controller labels a pod as exposed when it appears in an expose[] entry. It adds the egress label only when that pod sets egress: true in instancerConfig. Leave it false when the challenge does not need internet access.

Any pod with defined ports will be accessible across all pods part of the challenge instance by using the pod name directly as the DNS name, e.g. pod postgres can be connected to by using postgres:5432.

Exposed hostnames inside pods

The controller writes the resolved public endpoints onto every instance pod template as a JSON annotation named rctf.osec.io/exposed-hostnames. Challenge code can read that annotation through Kubernetes’ downward API when it needs to know its own public callback URL, WebSocket URL, or peer-facing hostname.

Each array entry follows the matching expose[] entry:

1
[
2
  {
3
    "kind": "https",
4
    "hostPrefix": "web-demo",
5
    "host": "web-demo-7f3a2c1d9b80.instances.ctf.example.com",
6
    "port": 443,
7
    "containerName": "app",
8
    "containerPort": 8080,
9
    "title": "Challenge"
10
  }
11
]

Expose it to a container as an environment variable with fieldRef:

1
instancerConfig:
2
  config:
3
    pods:
4
      - name: app
5
        spec:
6
          containers:
7
            - name: app
8
              image: ghcr.io/example/web-demo:latest
9
              env:
10
                - name: RCTF_EXPOSED_HOSTNAMES
11
                  valueFrom:
12
                    fieldRef:
13
                      fieldPath: metadata.annotations['rctf.osec.io/exposed-hostnames']

RCTF_EXPOSED_HOSTNAMES is a JSON string. Hidden endpoints (shouldDisplay: false) are included because they still exist for routing.

Per-pod safety checklist

Kubernetes does not expose every resource limit directly in a PodSpec, and the controller does not add limits to the pod definition you provide. Set the following values on every pod under instancerConfig.config.pods[].

#!/bin/sh
ulimit -n 1024
exec /your/challenge "$@"

RBAC and the rCTF service account

The example creates a single ServiceAccount named rctf in kube-system and a matching ClusterRole granting only what the API needs:

1
rule:
2
  api_groups: ['rctf.osec.io']
3
  resources: ['challengeinstances']
4
  verbs: ['create', 'get', 'delete', 'patch']

The rCTF API never reads or writes any other resource type. The kubernetes_secret_v1.rctf_token resource issues a kubernetes.io/service-account-token so the token doesn’t rotate. The value comes back through the rctf_instancer_auth_token Terraform output.

The controller’s RBAC configuration is under apps/k8s-operator/config/. It grants access to the namespaces, deployments, services, network policies, and Traefik resources needed to update each instance. The Traefik resources are IngressRoute, IngressRouteTCP, and Middleware. Run make manifests to generate the CRD in apps/k8s-operator/config/crd/bases/.

Example challenge config (Konata)

This complete Konata example is adapted from misc/pwnable-document-fabricator in the SekaiCTF 2026 challenge repository.

1
challenges:
2
  - category: misc
3
    name: pwnable document fabricator
4
    releaseTime: '2026-06-28T08:00:00Z'
5
    author: SekaiCTF
6
    description: |
7
      yep it's another web challenge.
8
    attachments:
9
      files:
10
        - "challenge/"
11
      exclude:
12
        - "challenge/readflag.c"
13
      additional:
14
        - path: readflag.c
15
          strContent: |
16
            #include <stdio.h>
17
            int main() { printf("SEKAI{dummy_flag}"); }
18
      stripComponents: 1
19
    flags:
20
      rctf: SEKAI{example}
21
    instancerConfig:
22
      instancer: k8s
23
      challengeIntegrationId: pwnable-doc
24
      timeoutMilliseconds: 1800000
25
      expose:
26
        - kind: https
27
          hostPrefix: pwnable-doc
28
          containerName: app
29
          containerPort: 8080
30
      config:
31
        pods:
32
          - name: app
33
            egress: false
34
            ports:
35
              - protocol: TCP
36
                port: 8080
37
            spec:
38
              restartPolicy: Always
39
              terminationGracePeriodSeconds: 0
40
              automountServiceAccountToken: false
41
              enableServiceLinks: false
42
              containers:
43
                - name: app
44
                  image: "{{ images['pwnable-doc'] }}"
45
                  ports:
46
                    - containerPort: 8080
47
                  volumeMounts:
48
                    - name: tmp
49
                      mountPath: /tmp
50
                  resources:
51
                    requests: { cpu: 100m, memory: 128Mi }
52
                    limits: { cpu: 500m, memory: 512Mi }
53
                  readinessProbe:
54
                    tcpSocket: { port: 8080 }
55
                    initialDelaySeconds: 5
56
                    periodSeconds: 3
57
                  securityContext:
58
                    readOnlyRootFilesystem: true
59
                    allowPrivilegeEscalation: false
60
                    capabilities:
61
                      drop: ["ALL"]
62
              volumes:
63
                - name: tmp
64
                  emptyDir:
65
                    medium: Memory
66
                    sizeLimit: 512Mi
67

68
deployment:
69
  images:
70
    - path: challenge
71
      name: pwnable-doc
72
      tag: latest
73
      registryName: instancer-challenges
74
      platform: linux/amd64

Things worth pointing at in this example:

• egress: false keeps the pod sealed off from public internet egress, though it can still connect to any ports part of the same challenge instance. Set it to true only for challenges that need outbound access.
• Resource requests and limits keep one instance from starving the node. Set both from the expected peak load for one team.
• readinessProbe keeps Traefik from routing to the pod before the app is up. Without it, the first request after creation often 502s while the container is still booting.
• readOnlyRootFilesystem plus the sized emptyDir gives the app a bounded writable /tmp/ without letting it write into the image layer.
• securityContext locks the container down with dropped capabilities and no privilege escalation.
• {{ images['pwnable-doc'] }} resolves to the fully-qualified registry path Konata pushed to (registries.instancer-challenges + the image name + tag).
• The additional attachment entry ships a dummy readflag.c while the real challenge flag is kept out of the handout.

For the rest of the Konata schema, see Konata.

Troubleshooting

Kubernetes events appear in kubectl describe challengeinstance <name>. Compare them with the controller logs from kubectl logs -n rctf-operator-system -l control-plane=controller-manager when the resources for an instance are not being created or removed correctly.