Skip to content

Release Promotion (shadow)

rig-conductor#2733 is step 3 of the plan in dashecorp/invotek: projects/invotek-prosjekt/proposals/2026-10-22-releases-and-promotion.md (invotek#79). The operator requires that a customer's Kundeadmin, or the operator, can send a tested version to production from prosjekt.invotek.no.

A site's deploy.yml builds one artifact per merge, deploys it to staging, and its Promote to production job waits on the repo's production environment. Today the operator approves that job in GitHub. Later, a new org-owned release App is the environment's only protection rule, and the conductor approves on a passkey-checked command from the portal.

This step is shadow. The conductor records what the release App reports and answers what waits. It never approves, rejects or cancels a deployment, makes no GitHub call, and holds no App key. invotek-no's production keeps the operator as its reviewer.

Routes

Route What it does
POST /api/webhook/release The release App's webhook. Verified with the App's own secret; see below.
GET /api/promotions?repo=owner/name {repo, waiting: [...], recent: [...]}: every waiting release, newest run first, and the ten most recently requested in any state.

Each release is {repo, runId, runUrl, sha, ref, environment, deploymentId, state, requestedAt, requests, supersededByRunId, supersededBySha, conclusion, completedAt}. state is waiting, superseded, deployed or failed. Times are ISO 8601 UTC, taken from GitHub's payloads.

GET /api/promotions takes a repo, not a tenant. The portal knows each project's deploy repo, which the operator sets on the project's card in /_admin (dashecorp/invotek-portal#223). The conductor resolves the repo's tenant itself, as everywhere else, and never takes a tenant id from a caller. An unknown repo answers empty lists. It sits behind Access like the conductor's other read routes. It is not GET /api/releases, which is the dashboard's release-notes list.

The portal reads it every minute (invotek-portal#225), for each deploy repo, with its own service token, which Access admits to this path only (infra#736). It copies the releases into its own D1 and lists them under «Versjoner» on the project's card, and it e-mails the operator once when a release starts waiting (invotek-portal#227). It refuses an answer in which any release differs from the shape above, and keeps what it had. invotek-portal docs/2026-10-29-versjoner.md has its side.

The webhook

Checks, in this order. Each refusal records nothing.

  1. The secret. RELEASE_APP_WEBHOOK_SECRET must be 32 to 256 printable ASCII characters with no whitespace (write the Secret with printf %s), and must not equal GITHUB_WEBHOOK_SECRET. That last rule keeps the release App apart from dev-e-bot: a delivery signed for the GitHub webhook never verifies here. Otherwise every delivery answers 503 not_configured, and the log names why (Missing, Malformed, ReusesGitHubWebhookSecret), never the value.
  2. The size. Over 512 KiB is 413 payload_too_large. At most one byte past the limit is read.
  3. The signature. X-Hub-Signature-256 exactly once, as sha256= and 64 lowercase hex, matching the HMAC-SHA256 of the exact body bytes in constant time. Anything else is 401 invalid_signature; the log says Missing, Malformed or Mismatch.
  4. The event. X-GitHub-Event is required (400 missing_event), and the body must be JSON (400 invalid_json).
  5. The payload (ReleaseWebhookPayload, Core). A signed delivery that is not the shape GitHub documents is 400 invalid_payload with the field.
  6. The repo must be a GitHub owner/name. It is lowercased.
  7. A sha must be 40 lowercase hex.
  8. The run id comes from deployment_callback_url, and only when it is exactly https://api.github.com/repos/{this repo}/actions/runs/{id}/deployment_protection_rule. The URL may have no other host, port, user info, query, fragment, extra segment or percent-encoding. Step 5 approves at that URL, so a URL for any other repo is refused here rather than trusted later.
  9. The tenant. It is resolved from the repo only. The release App's installation id names no tenant. An unresolved repo counts as tenant-0 only inside dashecorp (TenantMatch.CoalesceInDefaultOrg), and the tenant must have a registered database. Otherwise the answer is 200 {"recorded": false, "ignored": "no_tenant"}. A resolver failure is 503 tenant_unavailable.
Delivery Recorded
ping nothing; 200 {"pong": true}
deployment_protection_rule requested, environment production (any case) RELEASE_WAITING. A redelivery, or a late delivery of an older deployment, adds nothing (unchanged). A re-run's new deployment waits again.
workflow_job completed success, job named exactly Deploy to staging RELEASE_SUPERSEDED on each older run of the same repo that waits.
workflow_run completed RELEASE_DEPLOYED on success, otherwise RELEASE_FAILED with the conclusion. Only for a run that waited, and only when the run's updated_at is not before the request (an earlier attempt, delivered late, adds nothing).
anything else nothing; 200 with ignored and the reason

The answer for a recorded fact is {"recorded": true|false, "outcome": "recorded"|"unchanged"|"not_tracked", "repo", "runId"}. For a staging deploy it is {"recorded", "repo", "runId", "superseded": [run ids]}.

Why supersession is recorded

GitHub does not replace a waiting job. The promote job gets its place in the workflow's concurrency group before the approval wait, so a newer run's promote stays pending behind the waiting one and never reaches the gate. Seen on dashecorp/invotek-no on 2026-10-23: run 39767920473 waited while run 39929373887 sat pending, until the first was cancelled. The older run can no longer promote anyway, because its staging check stops it. In shadow the conductor records RELEASE_SUPERSEDED and pushes the Cockpit; the orchestrator cancels the old run by hand. Step 5 will cancel it.

So the release App must subscribe to workflow_job as well as deployment_protection_rule and workflow_run.

The release stream

One stream per run: release:<owner/repo>:<run id>, lowercase, in the repo's tenant database (MartenReleaseStore). Its events are typed records, which Marten names release_waiting, release_superseded, release_deployed and release_failed. ReleaseHistory (Core) is the pure fold and decides what each fact appends. The store re-reads the stream and decides again on every write, and retries a lost race up to three times. Every decision is idempotent and has no side effect. A stream holding an event the fold does not know fails closed.

release: is a reserved namespace, as cockpit-task: is. POST /api/events refuses a body streamId or agentId there (400), and SubmitEvent never appends there (reserved_stream). So no caller can forge a release event.

The Cockpit is pushed through the existing operator push relay (OperatorPushRelay): - «🚀 Release waiting» with the run link; - «⏭️ Release superseded» with the old run's link, which is the run to cancel.

How a run ends is not pushed. A failed promote already pages as a failed workflow.

What rig-gitops provisions

Provisioned 2026-10-28. The operator created the org-owned release App dashecorp-release (App id 5508310) from a manifest and installed it on dashecorp/invotek-no only (installation 176123484; dashecorp/infra BOOTSTRAP.md §26). Its key and webhook secret went straight from GitHub into rig-gitops' credential-ops environment, unseen. The credential-ops step release-app (run 40667375680, approved by the operator) encrypted them into the Secret rig-conductor-release-app, rig-gitops#1065:

Key Read by
webhook-secret this route, as RELEASE_APP_WEBHOOK_SECRET (optional: true)
private-key nothing yet; step 5's App token provider
app-id nothing yet; step 5, with the key

secretKeyRef values are read once, at container start. The route answers 503 until a pod starts after the Secret exists, and from then on it verifies signatures: an unsigned POST answers 401. Restart and rotation: rig-gitops docs/sops.md, «rig-conductor release App».

The Access bypass

Applied 2026-10-23 (dashecorp/infra#735), once this image ran. It covers rig-conductor.dashecorp.com/api/webhook/release only; GET /api/promotions stays behind Access, with the portal's own path-scoped token (infra#736).

What shadow sees on invotek-no

GitHub sends deployment_protection_rule only for an environment where the App is a protection rule. On invotek-no that is step 6, the switch. Until then this route receives workflow_job and workflow_run for invotek-no, and records no RELEASE_WAITING there; a finished run it did not see waiting is not_tracked. The whole flow, the protection rule included, is first exercised on a scratch repo with its own production environment (step 5).

So until step 6 the route answers empty lists for invotek-no, and the portal lists no versions for invotek.no and sends no e-mail about one.

Not built here

Step 5 builds them; its design is Promotion commands (rig-conductor#2760).

  • The App token provider, and every GitHub write: approving or rejecting at the callback URL, cancelling a superseded run (step 5).
  • The portal's commands (promote, rollback) and the job's OIDC check (step 5); confirm and the promotion policy per repo come with step 9. The commands' verifier is built in Core and not yet wired (rig-conductor#2762).
  • The portal's «Send til produksjon» and «Rull tilbake» (step 5). «Versjoner» and the deploy repo are built; passkeys come with step 9 (decided 2026-10-30).