Credentials on this page are redacted. This is the public export; the real values live in docs/ACCESS.md in the private repository.

The installation

What is running on this machine, and how it was built — README.md

OpenChoreo platform demo

A fully realized OpenChoreo installation on one machine, built to answer a platform engineer's question: how well does this standardize CI, CD and observability across applications?

Everything is pinned. OpenChoreo's API is still v1alpha1 and moved a great deal during 2026, so every version lives in platform/bootstrap/versions.env and the OpenChoreo repo itself is vendored at v1.2.3 under vendor/.

What is running

Five k3d clusters: one per OpenChoreo plane, plus a DR data plane, connected by mTLS WebSocket cluster-agents — the topology OpenChoreo calls "production-like".

Plus, on the host and deliberately outside OpenChoreo:

Getting at it

docs/ACCESS.md is the full map: every UI, credential, CLI and API, plus the optional Tailscale setup that replaces the port numbers with MagicDNS names.

docs/WORKING-ON-IT.md is the "I want to change X" guide: edit an app, change the CI pipeline, change the golden path, add a backing service, promote, roll back.

The short version:

export KUBECONFIG=~/.kube/openchoreo.yaml

open https://console.jung.town    # the console -- projects, deploys, builds,
                                  # traces, alerts, and the walkthroughs
open https://portal.jung.town     # the portal -- the catalog, and TechDocs
open https://grafana.jung.town/d/oc-delivery/delivery   # what is running, and what built it

Sign in with GitHub once and the portal, Grafana, Gitea, Zulip and Argo all accept the same session. Access is a GitHub allowlist — platform/bootstrap/thunder-scripts/66-allowlist.sh.

The applications are deliberately not published: being deployed is not the same thing as being on the internet. *.jung.town is a wildcard A record at the LAN address, so a name under it resolves for everyone and connects only from this network -- which means a component can look published, be linked from the dashboards, and still be reachable by nobody. A specific proxied CNAME in platform/openchoreo-config/public/dns-records.yaml is what actually puts something on the internet, and that file is the one place to look. Publishing one is an explicit public-endpoint resource, as the storefront does for shop.inkwell.jung.town. The full model, including which services the gateway authenticates on behalf of and why three are exempt, is in docs/ACCESS.md.

The repository itself

The whole thing lives in Gitea alongside the applications it deploys:

https://git.jung.town/platform-eng/openchoreo-demo

git clone it and run git submodule update --init — the twelve app and module repositories are submodules pointing at this same Gitea, pinned to the commits that are actually deployed. A plain clone is enough: main is the default branch and carries everything, and platform/observability-and-promotion is the same commit.

platform-eng/platform is a different thing and still exists: a subset of this tree pushed by sync-to-gitea.sh as a single commit with no history, so that builds can consume the Dagger modules and platform config over git. This repository is the history; that one is the feed.

One thing to know before reading the history. infra/.webhook-secret was committed twice, before .gitignore covered it — and .gitignore never untracks. The value is in two old commits and will stay there. It was rotated on 2026-09-03 (infra/rotate-webhook-secret.sh), so what is in history is a dead credential; rewriting 90 commits to remove it would change every SHA for no security gain. Nothing else was ever committed: .gitea-token, .github-oauth, .cloudflare-token, .zulip-announcer and tailscalecreds.env are gitignored and have never been tracked.

Standing it up

export KUBECONFIG=~/.kube/openchoreo.yaml

./platform/bootstrap/00-tooling.sh          # kubectl, helm, k3d, dagger, occ into ~/bin
sudo ./platform/bootstrap/01-host.sh        # inotify limits + dagger cache dir
docker compose -f infra/compose.yaml up -d  # gitea + registry caches
./infra/seed-gitea.sh                       # orgs, repos, admin token

./platform/bootstrap/10-clusters.sh         # 4x k3d (dp2 comes later, in 36)
./platform/bootstrap/20-prereqs.sh          # gateway-api, cert-manager, kgateway, ESO
./platform/bootstrap/21-control-plane.sh
./platform/bootstrap/22-data-plane.sh
./platform/bootstrap/23-workflow-plane.sh
./platform/bootstrap/24-observability-plane.sh
./platform/bootstrap/25-observability-modules.sh
./platform/bootstrap/26-lgtm.sh             # Loki, Tempo, Pyroscope, Grafana
./platform/bootstrap/27-alloy.sh            # the collector, on all four planes
./platform/bootstrap/28-beyla.sh            # eBPF instrumentation
./platform/bootstrap/29-externaldns.sh      # required for every jung.town name

./platform/bootstrap/30-dagger-engine.sh    # persistent engine + cache PV
./platform/bootstrap/31-build-cache.sh      # OpenChoreo's Zot cache (benchmark baseline B)
./platform/bootstrap/32-cnpg.sh             # CloudNativePG + extension image catalogs
./platform/bootstrap/33-flux.sh             # source + helm controllers; the helmchart resource type needs them
./platform/bootstrap/34-cilium.sh           # SEE THE WARNING BELOW — rebuilds the data plane
./platform/bootstrap/35-shared-secrets.sh   # the shared OpenBao and its per-cluster auth mounts
./platform/bootstrap/36-dr-data-plane.sh    # the fifth cluster, for the failover demo
./platform/bootstrap/40-platform-config.sh  # component types, traits, resource types, workflows

./platform/openchoreo-config/apply-coredns.sh  # host.k3d.internal + the localhost rewrites
./platform/bootstrap/restore-shared-secrets.sh --apply  # AFTER ANY REBOOT: the shared OpenBao is in-memory
./platform/loki-adapter/install.sh          # portal Runtime Logs + Traces tabs
./platform/finops-adapter/install.sh        # cost tab; the portal 500s without it
./platform/project-flow/install.sh          # review environments + promote-on-merge
./apps/console/install.sh                   # the console's credentials into the shared store
./platform/grafana-dashboards/apply.sh      # the 22 dashboards
./platform/zulip-realm/configure.sh         # the realm, streams and bot identities
./platform/techdocs/publish.sh              # tech docs from the live CRs
./platform/portal-image/build.sh            # the portal's own patches
./infra/git-credentials.sh                  # token out of every nested .git/config
./infra/setup-webhook.sh apps/inkwell               # push -> build
./platform/project-flow/setup-hook.sh apps/inkwell  # pull_request -> review environment
./platform/sync-to-gitea.sh --apply         # the platform's own source into Gitea

Every script is idempotent.

platform/bootstrap/thunder-scripts/* register the OIDC applications (GitHub IdP, Grafana, Zulip, Gitea, the gateway) and the GitHub allowlist; rerun-thunder-bootstrap.sh replays them all.

Stopping it, and starting it again

The demo costs about 24GB of the host's 31, nearly all of it the four k3d clusters. To get the machine back without losing anything:

./platform/capacity/power.sh status      # what is running, and what it costs
./platform/capacity/power.sh stop        # stop the four clusters (~24GB back)
./platform/capacity/power.sh start       # bring it back, in order, and repair

stop refuses while a build is running, and never removes a volume — Gitea is the git remote for every repository here and the five registry mirrors are what make a restart fast. It leaves infra/compose.yaml up for that reason; --all stops that too, at the cost of emptying OpenBao.

start is not the reverse of stop, because two things do not survive a container restart and neither announces itself: host.k3d.internal (below) and the in-memory secret store. The script runs both repairs, which is the whole reason to use it rather than four k3d cluster start commands.

This is a different tool from platform/capacity/hibernate.sh, which scales demo workloads to zero inside a running platform. That reclaims what the applications cost; this reclaims what the platform costs.

apply-coredns.sh is not optional, and a reboot needs it again. host.k3d.internal has to resolve for two different resolvers, and a reboot breaks both:

• Pods ask CoreDNS, and coredns-rewrite.yaml points every *.openchoreo.localhost name at host.k3d.internal — which is how the cluster agent on each plane reaches the control plane, and therefore how every deploy, promotion and rollback works. k3d injects the entry into the coredns ConfigMap at cluster creation, k3s owns that ConfigMap and re-templates it from the node list, so the entry is gone after a reboot. This script serves the name from coredns-custom instead, which k3s does not manage and which does survive.
• The kubelet reads the node container's /etc/hosts, and Docker rewrites that file on every container start, so nothing survives there. It matters more than it looks: every registry mirror in platform/bootstrap/k3d/config-*.yaml is spelled http://host.k3d.internal:<port>, so without the entry a node cannot pull any image — not the applications, not ghcr, not docker.io.

Neither failure names DNS. The first reports itself as no agents found for plane dataplane/default on every ReleaseBinding; the second as ImagePullBackOff on whatever restarted.

Install the unit so this happens on its own:

sudo cp platform/bootstrap/openchoreo-hostnames.service /etc/systemd/system/
sudo systemctl enable openchoreo-hostnames.service

Tailnet names are optional and are not set up here. docs/ACCESS.md describes a 05-tailscale.sh; that script is not in this repository and the Tailscale operator is not installed on any cluster. Public access is through Cloudflare (30-cloudflared.sh) and the stale *.ts.net kubeconfig contexts are left over from an earlier attempt. Nothing below depends on it.

34-cilium.sh destroys and rebuilds the data plane. Everything scripts 22–33 created on it goes with it. Run it in order, or not at all — discovering it late and running it then will take your databases with it.

30-cloudflared.sh and 30-dagger-engine.sh share a number, so lexical order runs cloudflared first. It is deliberately absent from the list above: it is only needed to publish something through the tunnel, and it fails closed without a Cloudflare token. See docs/ACCESS.md.

Without 40-platform-config.sh nothing in "What was built on top" exists — it applies every component type, trait, resource type and workflow the demo depends on.

01-host.sh is not optional. Ubuntu ships fs.inotify.max_user_instances = 128. Four k3s nodes need ~1024 and fail with a misleading "too many open files" otherwise.

What was built on top

OpenChoreo ships the primitives; these are the platform's opinions expressed in them.

platform/openchoreo-config/component-types/

ruby-service, ruby-worker, ruby-scheduled-task — generated from OpenChoreo's shipped types by generate.sh rather than hand-copied, so the ~270 lines of upstream HTTPRoute/visibility logic never drift. On top the platform adds:

The files are ruby-service-v1.yaml, ruby-service-v2.yaml and so on: every construct OBJECT carries its major version, including the first. The names above are the LOGICAL names, which is what a manifest writes and what the openchoreo.dev/name label carries — a consumer never names an object. See docs/VERSIONING.md.

• allowedWorkflows locked to rails-builder — the actual enforcement point. A component naming any other pipeline is rejected.
• mandatory liveness/readiness probes, non-root securityContext
• Rails-appropriate resource defaults (the shipped 256Mi OOMKills a Rails boot)
• the start command, because the workload descriptor schema cannot express one
• a validation refusing floating :latest tags

Ownership is enforced separately, in platform/openchoreo-config/policies/ — ComponentType validations cannot see a Component's own labels, so a ValidatingAdmissionPolicy rejects at kubectl apply instead. See docs/FINDINGS.md.

platform/openchoreo-config/traits/

• otel-instrumentation — patches OTLP endpoint, service name and resource attributes into the container. The collector address is a platform concern; no app repo contains it.
• slo-alerts — the two alerts the platform considers non-negotiable for a service, with per-environment thresholds.
• rails-runtime-secrets — generates a SECRET_KEY_BASE per component and environment on the data plane via an External Secrets password generator, so no team invents or commits one.

platform/openchoreo-config/resource-types/postgres.yaml

Replaces the shipped postgres type, which is a single-replica StatefulSet on an emptyDir and is explicitly labelled demo-only. This one is CloudNativePG with real storage, and exposes optional capabilities as a developer parameter:

parameters:
  database: inkwell
  extensions: [vector, timescaledb]   # pgvector, TimescaleDB

while instance count, storage and backups stay per-environment platform decisions. The output contract (host, port, database, username, password, url) is identical to the shipped type, so a component can be moved between them without touching its Workload.

platform/dagger-modules/

Reusable, versioned CI functions. See Build pipeline.

The applications

Three services in one project, deliberately built three different ways, so that one request crosses three languages and two kinds of packaging.

Two more projects exist to prove the platform works for software nobody here wrote:

Which door a piece of software goes through is decided by its packaging rather than by anything meaningful: an image can consume the catalog's Postgres, a chart cannot, because a Resource cannot read another Resource's outputs. That asymmetry is documented in docs/FINDINGS.md.

They are all in the inkwell project because the portal groups traces per project — split them up and the waterfall splits too.

browser → kgateway → inkwell-web ──enqueue──→ inkwell-worker ──HTTP──→ inkwell-enrich ──S3──→ Garage
                          │                        │                        │
                          └──────── Postgres ──────┘                        └── Postgres

One trace covers that whole path. Rails and FastAPI each propagate W3C traceparent; the collector derives the component identity from pod labels, so no application sets a resource attribute to be attributed correctly. docs/WORKING-ON-IT.md has the query to pull one out by hand — and the reason a trace that stops at a service boundary is usually the collector's sampler rather than your code.

apps/inkwell — a Rails 8 notes app, deployed as three components sharing one image (web, Solid Queue worker, nightly task).

It exists to make the optional database capabilities observable: it detects what the platform actually provisioned and says so.

CI runs the suite twice, against a Postgres that has the extensions and one that does not, so "degrades gracefully" is verified rather than asserted.

Build pipeline

The platform's Rails pipeline is one ClusterWorkflow (rails-builder) whose build step calls a Dagger module:

checkout-source -> dagger-build -> generate-workload-cr

where dagger-build runs lint, dependency audit, the test suite in both capability modes, then builds and publishes.

Two structural differences from the stock dockerfile-builder:

1. One long-lived engine, not a cold builder per run. The Dagger engine is a StatefulSet whose cache is a retained PV on the host NVMe (/var/lib/openchoreo-dagger), so it survives k3d cluster delete.
2. No tar round trip. Stock runs podman build → podman save to a tar on a shared PVC → podman load in a second privileged pod → podman push. Dagger pushes the layers it already has. Privilege lives once, in the engine, instead of in every build pod.

Call the same pipeline from a laptop:

kubectl port-forward -n openchoreo-workflow-plane svc/dagger-engine 18080:8080 &
export _EXPERIMENTAL_DAGGER_RUNNER_HOST=tcp://127.0.0.1:18080

cd platform/dagger-modules
dagger call ci --source=../../apps/inkwell/rails \
  --registry=host.k3d.internal:10082 --image=inkwell --tag=dev

See bench/ for the A/B/C measurements.

Verified end to end

A single git push to apps/inkwell:

1. Gitea fires a push webhook at the OpenChoreo API.
2. Auto-build matches the changed paths against each component's appPath and creates three WorkflowRuns, one per component.
3. Each runs rails-builder: rubocop, bundler-audit, RSpec against Postgres with the optional extensions and again without, then build and publish. ~90s each, in parallel, against the shared warm engine.
4. generate-workload writes each component's Workload from its deploy/<component>/workload.yaml with the freshly published digest.
5. autoDeploy rolls all three out; data survives the rollout.

Observed afterwards:

$ curl -H 'Host: staging-inkwell.jung.town'     http://localhost:19080/inkwell-web-http/health/ready
{"status":"ready",
 "capabilities":{"vector":true,"timescale":true,"continuous_aggregates":false},
 "search_mode":"semantic"}

• pgvector HNSW search returns the right note; the worker embedded all of them through Solid Queue
• activity_events is a TimescaleDB hypertable; the hourly rollup correctly reports on_the_fly because the Apache build has no continuous aggregates
• the nightly ScheduledTask runs and emits a structured digest
• log streams in Loki, 14 Prometheus targets
• one trace, 68 spans, 3 components: POST /notes on inkwell-web → Solid Queue publish → inkwell-worker consume → POST /embed and POST /attachments/{note_id} on inkwell-enrich → S3.PutObject on Garage

See docs/FINDINGS.md for everything that broke on the way and why.

Observability

Grafana LGTM + Pyroscope + eBPF, the only observability stack. OpenChoreo's shipped OpenSearch was removed: it cost roughly 4GB to answer questions Loki and Tempo already answer. The portal keeps all three of its tabs — Metrics from Prometheus, and Runtime Logs and Traces through two small adapters that translate the Observer's contract onto Loki and Tempo (platform/loki-adapter/, ~64Mi for both). Full detail in docs/OBSERVABILITY.md.

Following the chain. Commit, build, deployed URL and the UIDs that logs, traces and profiles key on all existed on the control plane and nothing joined them. project-flow publishes that join as a Prometheus _info series, which is what lets two dashboards answer the questions worth asking:

• Delivery — https://grafana.jung.town/d/oc-delivery/delivery — every deployment, each row linking to the commit in Gitea, the build in Argo, and the running URL.
• Component 360 — https://grafana.jung.town/d/oc-component-360/component-360 — one component in one environment, with what is deployed, what built it, and its metrics, logs, traces and profiles on one screen.

The migration was one line — the otel-instrumentation trait's collector endpoint — after which Alloy fans every span out to both Tempo and OpenChoreo's collector. No application repo changed. That is the case for the collector address being a platform concern.

Two pipelines, and nothing live until it is promoted

pull request  ->  review-pr-N-<project>.jung.town   ephemeral, torn down on merge
merge to main ->  staging-<project>.jung.town       automatic
promote       ->  production-<project>.jung.town    explicit

The second exists because a staging Outline is not a rehearsal for the production Outline, it is a second, empty wiki. Modelling stateful platform services as applications means either paying for a duplicate of each or leaving half the environments permanently dark.

Hostnames are project-scoped, and published names are environment-scoped: shop.inkwell.jung.town is production, shop-staging.inkwell.jung.town is staging. A deployment that was never promoted cannot occupy the production name — which it could, and did, when a public-endpoint in development owned it.

Creating a project

./platform/templates/new-project.sh --template rails-service --name bakery

The repository, the source, the webhook, the Project, its cell, its database and its components. Four templates: rails-service, static-site, zulip-bot, ruby-gem.

rails-service produces three components from one repository through two pipelines — a Rails service and its worker from /rails, and a documentation site from /docs — and ships endpoints that make each kind of slowness recognisable in a flame graph. ruby-gem publishes a gem to Gitea's registry and a site of its generated docs, with only the site on a pipeline, because a gem version is immutable and republishing one on every merge would make that mistake automatic.

Asking the platform from chat

@openchoreo status inkwell        what is deployed where, and what built it
@openchoreo where inkwell-web     every environment, and whether they agree
@openchoreo promote showcase      staging -> production

Every reply carries links to the trace for the question you just asked, the commit that built what is running, and the component's dashboards. Reads come from the same delivery index the dashboards use, so the bot cannot disagree with them; writes go through the same endpoint the CLI and the PR label use.

Platform activity is announced into a Zulip stream per project, topic per activity kind, with non-production events diverted to #platform-sandbox.

Component types the platform offers

Three Dagger modules in the same shape (ruby, python, frontend) sit behind them, so adding a language is a module and a module-function, not a new pipeline.

Zulip bots, provisioned end to end

apps/zulip-bots — two bots, one repository, one image. DM either from Zulip:

now Berlin                      # weather-bot, via Open-Meteo
add Release notes | shipped it  # inkwell-bot, writes to the notes service

The interesting part is that nobody creates the bot. A zulip-bot Resource runs a Job in the data plane that creates the account through Zulip's API and writes the key straight to a Secret; the component consumes it through envBindings exactly like a database. No human ever sees the API key.

Secrets, promotion, failover

• Secrets three ways — generated in the data plane for applications, a deploy token for CI, and one shared OpenBao with a per-cluster Kubernetes auth mount so a secret written once is readable from several clusters, each authenticating as itself. docs/SECRETS.md
• Promotion — the same immutable ComponentRelease runs in staging and production; only the environment configs differ (2 and 3 database instances). A merge reaches staging by itself, because staging is the pipeline root and that is what autoDeploy feeds. Production is an explicit act with three spellings — a CLI, a promote label on the pull request, and a chat command — all of which POST to the same endpoint, because only doing all four of its steps actually deploys anything.
• Failover — CloudNativePG promotes a replica in ~36s with no user-visible error; cross-cluster application failover takes ~60s by patching ReleaseBinding.spec.state. platform/failover/ is candid about the data story it does not solve.

Five clusters now: control, data, workflow, observability, and a DR data plane that deliberately runs a different CNI to prove a data plane is a contract.

Layout

Every directory under apps/, plus platform/dagger-modules, is a nested repository with its own Gitea origin — that is the point, since CI is webhook-driven. .gitmodules records all eleven, so git clone --recurse-submodules gets a complete tree on this machine. It does not anywhere else — see What a clone from GitHub gets.

infra/                    gitea, openbao + registry caches (compose), seed,
                          webhook and git-credential scripts
docs/                     ACCESS, OBSERVABILITY, SECRETS, WALKTHROUGH,
                          WORKING-ON-IT, FEEDBACK, FINDINGS
  screenshots/            the 53 captures the guides are illustrated with
site/                     all of the above as static HTML, generated by
                          platform/dev/build-docs-site.py and COMMITTED --
                          21 guides, 11 reference pages, 3 ledgers, 58 images
bench/                    build pipeline benchmark harness and results
vendor/openchoreo/        OpenChoreo v1.2.3, vendored (the source for the
                          generated component types and workflows)

apps/                     the demo applications, one Gitea repo each
  inkwell/                Rails 8: web + Solid Queue worker + nightly cron,
                          THREE components sharing ONE image and one build
  enrich/ ledger/         FastAPI and gRPC services inkwell calls
  storefront/             the one publicly published app (shop.inkwell...)
  chess/                  bot + engine + render; engine is unreachable by
                          design, which is why the conductor exists
  rideshare/              the only Go service, so the only memory/goroutine
                          profiles
  brochure/               scaffolded BY the platform, by the walkthrough
                          rather than by hand
  zulip-bots/             five bots, ONE image and one build; conductor.py is
                          the traffic generator for all of them
  zulip/                  vendored service, manifests only

platform/
  bootstrap/              numbered, idempotent install scripts + versions.env,
                          k3d configs, helm values, thunder-scripts/
  openchoreo-config/      THE GOLDEN PATH: component types, traits, resource
                          types, project types, deployment pipelines,
                          admission policies, CI workflows
  dagger/                 engine values + the retained cache PV
  dagger-modules/         the reusable CI module: platform, ruby, python,
                          frontend, gem-publish
  templates/              self-service scaffolding + new-project.sh, promote.sh
  project-flow/           the custom controller: webhooks, review
                          environments, promotion, scaffold and teardown
  grafana-dashboards/      22 dashboards in ci/, openchoreo/, platform/,
                          profiling/
  loki-adapter/           the portal's Runtime Logs and Traces tabs
  finops-adapter/         the cost tab
  portal-image/           Backstage patches (entity links, Grafana plugin)
  portal-catalog/         platform services as portal entities
  techdocs/               mkdocs generated from live CRs
  pylib/ gems/            the telemetry libraries the apps import
  capacity/               hibernate.sh, prune.sh — what to turn off, and how
  failover/               CNPG replica promotion across clusters
  demo-traffic/           continuous synthetic load, so no panel is empty
  zulip-realm/            realm, streams and bot identities
  sync-to-gitea.sh        push the platform's own source to Gitea

What a clone from GitHub gets

This repository is mirrored to a private GitHub repo, TeamTeaTime/openchoreo-profiling-demo. That mirror is the superproject only. Be clear about what does and does not survive the trip.

The eleven submodules do not. Every URL in .gitmodules is http://localhost:3001/... — a Gitea that runs on this host and is deliberately outside the clusters. On any other machine:

$ git submodule update --init
fatal: unable to access 'http://localhost:3001/apps/brochure.git/': Failed to connect

A clone gets eleven empty directories and the commit each one is pinned to. This is intended: the mirror exists to preserve the platform — the bootstrap scripts, the golden path, the documentation and the site — not to be a working installation somewhere else. apps/ is around 705 MB of real content here and zero bytes to anyone else.

Everything else does. docs/, site/, bench/, infra/, and all of platform/ except the two submodules (dagger-modules, openchoreo-config) are ordinary committed files and are complete in the mirror. In particular the whole documentation set is readable straight from a clone: open site/index.html.

Two files hardcode this machine and must be edited before anything is stood up elsewhere:

platform/bootstrap/openchoreo-hostnames.service likewise carries an absolute WorkingDirectory and PATH.

The credentials in docs/ACCESS.md are real for this installation — the Gitea account, the four Thunder seed logins, Grafana's admin:REDACTED. They reach services behind a Cloudflare tunnel and a tailnet, so they are useless without that network, but they are not placeholders. This is why the published site is gated (below) rather than served openly, and why the GitHub repo is private.

The published site

site/ is generated by platform/dev/build-docs-site.py and committed, so publishing it needs no build. It cannot be built in CI by construction: the generator imports the console's own Markdoc renderer out of apps/console and copies that submodule's WebP, and neither is reachable from a runner. Rebuild it here and commit the result:

platform/dev/build-docs-site.py --check    # fail if site/ is out of date
platform/dev/build-docs-site.py            # rewrite it

Publishing happens from this host, not from CI. The Cloudflare token reaches the whole jung.town zone -- DNS and Tunnels -- so copying it into GitHub Actions secrets to publish a static site is a bad trade. It stays in infra/.cloudflare-token.

platform/dev/publish-site.sh --dry-run    # run every check, deploy nothing
platform/dev/publish-site.sh              # check, then deploy

The script re-runs --check, repeats the structural guards CI applies, creates the Pages project if it is missing, reads the project's real *.pages.dev hostname from the API rather than assuming it, verifies the redactions, deploys by Direct Upload, and then fetches a redacted page back to confirm a placeholder is what got served.

.github/workflows/publish-docs.yml keeps the structural checks and attaches site/ to every run as an artifact, so anyone with repo access can read the docs without a Cloudflare login. It does not deploy.

The published site is redacted; this repository is not

The site is public. This repository is private, and docs/ACCESS.md stays a complete credential map — that is its job, and a private repo is the right place for it. The static export is a different artifact: build-docs-site.py replaces every credential with a placeholder at render time and fails the build if one survives, so the committed site/ and the CI artifact are both safe by construction.

What is replaced, and what is not:

Pages that had something replaced say so, in a note at the top.

Three checks, deliberately not the same check:

• build-docs-site.py fails the build if a listed value reaches site/. It also prints a REVIEW list of credential-shaped strings no redaction covers — advisory, because a denylist only removes what somebody listed. That review list is how the Grafana break-glass was caught: it is spelled three ways across two documents, and matching admin:REDACTED found none of them.
• publish-site.sh re-verifies the committed site/ immediately before uploading, by calling build-docs-site.py --verify-published so the values live in exactly one place. There is no flag to skip it. After deploying it fetches a redacted page back and confirms a placeholder is what got served.
• The GitHub workflow checks the redaction note is still on the four pages that carry one. It does not re-list the credentials: writing them into a workflow file would copy them into the artifact most likely to be read by someone who should not have them.

platform/dev/build-docs-site.py --verify-published   # check site/ as committed

Teardown

k3d cluster delete openchoreo-cp openchoreo-dp openchoreo-wp openchoreo-op
docker compose -f infra/compose.yaml down       # add -v to drop gitea + caches

The Dagger cache at /var/lib/openchoreo-dagger deliberately survives this. Remove it by hand if you want a genuinely cold rebuild.