Kubernetes

Last updated on November 20, 2026

The tofupilot Helm chart runs a self-hosted TofuPilot instance on your Kubernetes cluster. It runs the same services and images as the Docker Compose install, with your cluster's ingress controller in place of the bundled reverse proxy. One release is one instance, and one instance serves one organization.

What the chart runs

ComponentKubernetes objectRole
DashboardDeployment, 1 replicaTofuPilot application and API. Backs up and migrates the database when a new version starts.
PostgreSQLStatefulSetDatabase. Optional: you can use your own.
File storageStatefulSetSeaweedFS, S3-compatible, signed requests only. Attachments, uploaded files and build artifacts.
RealtimeDeploymentCentrifugo WebSocket server: live station status, telemetry, build logs.
Build workerDeployment, privilegedBuilds Deployments in its own Docker daemon. Optional.
Bucket jobJob, after each install and upgradeCreates the two storage buckets.

Resource names start with <fullname>: the release name, followed by -tofupilot unless the release name already contains it. With the release name tofupilot used on this page, <fullname> is tofupilot: the dashboard Deployment is tofupilot-dashboard and the database pod is tofupilot-pg-0.

Requirements

RequirementDetails
Kubernetes1.27 or later, linux/amd64 nodes
Helm3.14 or later
Ingress controllerAny. Tested with the official Traefik chart. See Ingress.
TLSA certificate for three hostnames, or cert-manager to issue them
StorageA default StorageClass with ReadWriteOnce volumes: 20 GB for the database, 50 GB for files, 10 GB for backups, 20 GB for the build cache (all configurable)
CNIOne that enforces NetworkPolicy (Cilium, Calico, ...). Otherwise set networkPolicy.enabled: false.
Pod SecurityThe build worker needs a privileged pod. If your policy forbids it, disable the build worker.
Sign-inSMTP or an OAuth provider, or nobody can create the first account
LicenseA license key from TofuPilot Orbit

You also need three hostnames pointing at your ingress controller: one for the app, one for file storage and one for realtime, for example tofupilot.example.com, storage.tofupilot.example.com and realtime.tofupilot.example.com. Browsers and stations reach all three.

Install

Get registry credentials

The chart and its images are in TofuPilot's registry, ghcr.io/tofupilot. Exchange your license key for read credentials:

curl -s -X POST https://www.tofupilot.com/api/docker/token \-H 'Content-Type: application/json' \-d '{"licenseKey":"lk_..."}'

The response holds a username and a token. Log Helm in with them:

helm registry login ghcr.io --username <username> --password <token>

Create the namespace and its secrets

Keep credentials out of your values file. Create the namespace, the image pull secret, and a secret for your sign-in credentials:

kubectl create namespace tofupilotkubectl -n tofupilot create secret docker-registry tofupilot-registry \--docker-server=ghcr.io --docker-username=<username> --docker-password=<token>kubectl -n tofupilot create secret generic tofupilot-auth \--from-literal=EMAIL_SMTP_PASSWORD=<smtp-password>

tofupilot-auth holds the secret half of each sign-in method you use: EMAIL_SMTP_PASSWORD, AUTH_GOOGLE_SECRET, AUTH_MICROSOFT_ENTRA_ID_SECRET, AUTH_GITHUB_SECRET, AUTH_GITLAB_SECRET or AUTH_ATLASSIAN_SECRET. The variables are the same as in the Docker Compose configuration.

Write your values file

This example uses Traefik and cert-manager. Adapt the hostnames, the ingress class, the issuer and the sign-in settings:

domains:app: tofupilot.example.comstorage: storage.tofupilot.example.comrealtime: realtime.tofupilot.example.comimagePullSecrets:- name: tofupilot-registryingress:className: traefikannotations:  cert-manager.io/cluster-issuer: letsencryptnetworkPolicy:ingressNamespaceSelector:  matchLabels:    kubernetes.io/metadata.name: traefik# Sign-in: SMTP here. Its password is in the tofupilot-auth secret.env:EMAIL_SMTP_HOST: smtp.example.comEMAIL_SMTP_PORT: "587"EMAIL_SMTP_USER: noreply@example.comEMAIL_FROM_AUTH: noreply@example.comextraEnvFrom:- secretRef:    name: tofupilot-auth

Save it as values.yaml. Every option is documented in the chart's own values:

helm show values oci://ghcr.io/tofupilot/charts/tofupilot

Install the chart

helm install tofupilot oci://ghcr.io/tofupilot/charts/tofupilot \--namespace tofupilot -f values.yaml \--set licenseKey=lk_... \--wait --timeout 65m

The first start applies every database migration, which takes a few minutes. Follow it with:

kubectl -n tofupilot rollout status deployment/tofupilot-dashboard

Pin a chart version with --version. Each chart version defaults to one TofuPilot release; set image.tag to run another one. The build worker needs 2.52.9 or later on Kubernetes: earlier images cannot start build containers there.

Do not pass --atomic (or --rollback-on-failure). If the migrations outlast the timeout, Helm would roll back to the previous version on a database that is already migrated. With --wait, give --timeout at least dashboard.startupSeconds, which defaults to one hour.

Create the first account

Open https://tofupilot.example.com and sign in with the method you configured. The first account becomes the organization admin.

If the sign-in page only offers single sign-on, no sign-in method is configured: check the env and extraEnvFrom settings. The install notes warn about it when env sets none.

Point your stations at the instance

Stations, scripts and the CLI use the app hostname, the same way as a Docker Compose instance. See Upload runs.

Ingress

The chart creates one Ingress per hostname. ingress.annotations applies to all three, and ingress.app.annotations, ingress.storage.annotations and ingress.realtime.annotations to one each. On the same key, the per-host value wins.

Two kinds of traffic need more than the usual defaults:

  • Uploads go straight from browsers and stations to the storage hostname, through presigned links: attachments up to 95 MB, and deployment bundles. The request body size limit and the request read timeout must allow them over your slowest link.
  • Realtime keeps a WebSocket open for as long as the dashboard or a station is connected. The idle timeout must allow it.
ControllerWhat to set
Traefik (v3)No body size limit. Each request has 60 s to arrive, so a slow upload fails with 502 Bad Gateway. Raise it on the Traefik chart: --set ports.websecure.transport.respondingTimeouts.readTimeout=600s. Measured: a 95 MB upload at 500 KB/s got a 502 at 60 s, and completed in 195 s with that setting. Idle WebSockets are not affected.
ingress-nginxThe chart's defaults lift the 1 MB body limit and the 60 s WebSocket timeout. ingress-nginx is retired upstream: prefer another controller for a new cluster.
GKE IngressAttach a BackendConfig with timeoutSec: 3600 to the realtime Service.
AWS Load Balancer ControllerSet alb.ingress.kubernetes.io/load-balancer-attributes: idle_timeout.timeout_seconds=3600.

TLS

Each Ingress reads its certificate from a secret named <fullname>-tls-app, -storage and -realtime. A cert-manager issuer annotation, as in the example values, creates the three.

To use one wildcard or multi-domain certificate instead, create its secret, set ingress.tls.existingSecret to its name, and remove the issuer annotation.

If the certificate comes from a private CA, stations must trust it. See Private CA, and publish the bundle with the chart's privateCa values.

Network policies

The chart's NetworkPolicies are on by default. They give the cluster the isolation the Docker Compose install gets from its private network:

  • PostgreSQL accepts connections from the dashboard and the build worker only.
  • The build worker, which runs your procedure code, reaches the stack's own services, DNS and the internet. It cannot reach link-local addresses, which covers every major cloud's metadata endpoint, nor 100.100.100.200. It has no IPv6 egress.
  • The dashboard, storage and realtime accept traffic from your ingress controller's namespace and from the components that call them, once networkPolicy.ingressNamespaceSelector is set: the dashboard from realtime and the build worker, storage from the dashboard, the build worker and the bucket job, realtime from the dashboard and the build worker.

Set networkPolicy.ingressNamespaceSelector to your ingress controller's namespace, as in the example values. The build worker reaches its own dashboard through the public hostname, and many clusters route it straight to the controller's pods. Without the selector, builds can fail on those clusters. networkPolicy.ingressControllerPorts lists the ports those pods listen on; the default covers ingress-nginx (80, 443) and Traefik (8000, 8443).

The policies are tested on Cilium, with Traefik in its own namespace. On other setups:

SetupWhat to change
CalicoAdd your pod CIDR to networkPolicy.deployerEgressExcept so builds cannot reach other pods.
OpenShiftSet networkPolicy.dns to the dns-default pods on port 5353.
NodeLocal DNSCacheAllowed by default (169.254.20.10).
Host-network controllers, AWS ALB in IP mode, GKE NEGLeave ingressNamespaceSelector empty: their traffic does not come from a namespace.
Git server inside the clusterAllow it in networkPolicy.deployerExtraEgress: builds clone from it.

Database and secrets

Bring your own PostgreSQL

Set postgresql.enabled: false and give the connection URL, either directly in postgresql.externalUrl or in a secret named by postgresql.externalUrlSecret, under the key DATABASE_URL. The database must run PostgreSQL 17 with pg_stat_statements in shared_preload_libraries, and its user needs the CREATE privilege on the database. If it runs inside the cluster, allow the build worker to reach it in networkPolicy.deployerExtraEgress: the network policies only open the bundled database.

Generated secrets

On the first install, the chart generates the instance secrets (session, database, storage, realtime and encryption keys) into the secret <fullname>-secrets, and reads them back on every upgrade. The secret is kept when you uninstall the release.

Back up <fullname>-secrets with the database. SECRETS_MASTER_KEY encrypts every stored config value and secret: without it they are unreadable.

Argo CD and other tools that render with helm template cannot read the secret back, and would generate new values on every sync. Use existingSecret with them: create the secret yourself, with the keys listed in the chart's README (helm show readme oci://ghcr.io/tofupilot/charts/tofupilot). Flux runs a real Helm install and needs nothing special.

Upgrade

Upgrade with your values file:

helm upgrade tofupilot oci://ghcr.io/tofupilot/charts/tofupilot \--namespace tofupilot -f values.yaml --version <chart-version>

Or reuse the values stored in the release, with the new chart's defaults:

helm upgrade tofupilot oci://ghcr.io/tofupilot/charts/tofupilot \--namespace tofupilot --reset-then-reuse-values --version <chart-version>

Do not use --reuse-values. It keeps the previous chart's defaults, security fixes included: an upgrade that way once kept an older, wider network rule for the build worker.

When a new version starts, the dashboard dumps the database, then migrates it. Downgrades are refused, because an older version cannot run on a migrated database. To go back, restore the dump instead (see Back up and restore). If an upgrade went to a version that never started (wrong tag, image pull error), go back with helm rollback.

A change to a secret listed in extraEnvFrom does not restart the dashboard. Restart it yourself:

kubectl -n tofupilot rollout restart deployment/tofupilot-dashboard

Back up and restore

Back up on a regular schedule:

WhatWhere
DatabaseThe volume data-<fullname>-pg-0, or your own PostgreSQL
FilesThe volume data-<fullname>-storage-0
Instance secretsThe secret <fullname>-secrets

Each time a new version starts, the dashboard also dumps the database into the <fullname>-backups volume, mounted at /backups in its pod. The naming, the 7-day retention and REQUIRE_BACKUP are the same as on Docker Compose: see Automatic pre-migration dumps.

To restore a dump named <date>_<time>_pre-v<new>_from-v<old>.sql.gz, run the version it came from, <old>. For a from-unknown or older-style dump, find <old> in the APP VERSION column of helm history tofupilot -n tofupilot. With your own PostgreSQL, run steps 2 and 3 against your server.

Copy the dump out of the dashboard pod:

kubectl -n tofupilot exec deploy/tofupilot-dashboard -- ls -l /backupsPOD=$(kubectl -n tofupilot get pod -l app.kubernetes.io/component=dashboard -o name | cut -d/ -f2)kubectl -n tofupilot cp "$POD:/backups/<file>.sql.gz" ./<file>.sql.gz

Stop the dashboard and the build worker (leave the build worker out if you disabled it), then replace the database with an empty one:

kubectl -n tofupilot scale deployment/tofupilot-dashboard deployment/tofupilot-deployer --replicas=0kubectl -n tofupilot exec tofupilot-pg-0 -- dropdb -U tofupilot tofupilotkubectl -n tofupilot exec tofupilot-pg-0 -- createdb -U tofupilot tofupilot

Load the dump:

gunzip -c <file>.sql.gz | kubectl -n tofupilot exec -i tofupilot-pg-0 -- psql -U tofupilot -d tofupilot -v ON_ERROR_STOP=1

Start the version the dump came from. image.allowDowngrade lets this one upgrade go back, and Helm scales both deployments back up:

helm upgrade tofupilot oci://ghcr.io/tofupilot/charts/tofupilot \--namespace tofupilot --reset-then-reuse-values \--set image.tag=<old> --set image.allowDowngrade=true

--reset-then-reuse-values keeps both values on every later upgrade. Clear them on the next one, or it stays on <old> with the downgrade guard off:

helm upgrade tofupilot oci://ghcr.io/tofupilot/charts/tofupilot \--namespace tofupilot --reset-then-reuse-values --version <chart-version> \--set image.tag=null --set image.allowDowngrade=false

With a values file, run the restore with -f values.yaml instead: the next -f values.yaml upgrade drops both.

Build worker

The build worker builds deployment bundles in its own Docker daemon, so its pod is privileged. A container escape from it is root on its node: give it a dedicated node pool with deployer.nodeSelector and deployer.tolerations.

If your Pod Security policy forbids privileged pods, set deployer.enabled: false. Everything else keeps working; only the Deployments build pipeline is unavailable.

Uninstall

helm uninstall tofupilot --namespace tofupilot

The secret <fullname>-secrets, the backups volume and the database and file volumes are kept, so a new install with the same release name finds its data again. To remove everything, delete the namespace after the uninstall:

kubectl delete namespace tofupilot

Deleting the namespace deletes the database, the uploaded files and the instance secrets. It cannot be undone.

Troubleshooting

SymptomCause and fix
Pods stuck in ImagePullBackOffThe pull secret is missing or the token is wrong. Check imagePullSecrets and get a new token from your license.
The sign-in page only offers single sign-onNo sign-in method is configured. Set SMTP or an OAuth provider in env and extraEnvFrom.
Uploads fail with 502 after about 60 sThe ingress read timeout. See Ingress.
Live views do not updateThe realtime hostname does not reach the realtime pod, or its WebSocket is cut by an idle timeout. Check the realtime Ingress and its certificate.
Builds fail to clone or to reach the dashboardNetworkPolicy. Set networkPolicy.ingressNamespaceSelector, and allow your Git server in deployerExtraEgress.
helm upgrade refuses: "older than the deployed"The downgrade guard. Restore a dump instead; use helm rollback only for a version that never started.

How is this guide?

On this page