ADR 0024: Airflow executable init-fetch uses a strict v2 wire boundary

• Status: Accepted
• Date: 2026-07-19
• Amends: ADR 0007, ADR 0008, ADR 0009

Context

The Airflow deployment-index v1 contract was introduced for local, parse-safe projection. It can describe runtime_artifact_delivery.mode: init_fetch, but it does not carry the complete executable image, artifact, trust-policy, and workload-pack contract required to run that mode safely. Treating an incomplete v1 declaration as runnable would let the provider fall back to scheduler-copied commands or inline bootstrap content.

The hardening increment needs one unambiguous executable wire while preserving the existing v1 local-preview journey. It also needs to state the actual trust boundary. Fixed pod commands prevent a workload pack from replacing provider owned execution fields; they do not make a compromised scheduler, provider, or local deployment cache harmless.

Decision

Freeze the wire and migration boundary

The provider applies this closed policy before installing any DAG:

Index wire	Delivery mode	Result
dpone.airflow-deployment-index.v1	local_preview	Supported non-runnable preview compatibility lane.
dpone.airflow-deployment-index.v1	init_fetch	Reject with DPONE_RUNTIME_ARTIFACT_DELIVERY_MIGRATION_REQUIRED; regenerate an immutable v2 deployment.
dpone.airflow-deployment-index.v2	init_fetch	The only strict executable indexed lane.
v2	Any other known mode	Reject with DPONE_RUNTIME_ARTIFACT_DELIVERY_MODE_UNSUPPORTED.
Either wire	Unknown mode or malformed field	Reject as an index field/schema error.

Strict fields are not retroactively required in v1. A v1 deployment is never silently reinterpreted as v2, and operators never repair an immutable index in place.

Make the executable path explicit

The only indexed executable path is:

producer
  -> immutable release/deployment
  -> airflow-deployment-index.v2
  -> trusted local cache/current snapshot
  -> lightweight provider
  -> KubernetesPodOperator
  -> dpone airflow runtime-init-fetch
  -> runtime-fetch-ready.json
  -> dpone airflow runtime-pack-exec
  -> verified workload argv

The v2 index carries an exact digest-pinned OCI runtime image, exact release/deployment/workload descriptors with positive byte counts, a workload identity, bounded ConfigMap references, and one immutable delivery context. The provider derives a canonical plan no larger than 16 KiB, transports it as DPONE_INIT_FETCH_PLAN_B64, and pins its digest in DPONE_INIT_FETCH_PLAN_SHA256 and the pod annotation.

The provider replaces pack-owned executable fields with these fixed commands:

init: dpone airflow runtime-init-fetch
base: dpone airflow runtime-pack-exec

Both containers use the exact same digest-pinned image. The base launcher revalidates the ready manifest and fetched pack, selects a structured argv from that pack, and replaces itself with the child process without invoking a shell. The ready manifest binds the pinned trust-policy digest, the effective attestation requirement selected by init, and its attestation status. The base launcher validates that decision against the same plan and does not reread the init-only registry or trust-policy ConfigMap. The child exit code therefore remains the KPO result.

Use deployment-set v2 as the runtime receipt

The exact fetched dpone.deployment-set.v2 is the runtime deployment receipt. The runtime does not generate or consume a second compact receipt schema. It verifies the fetched bytes against the plan descriptor, recomputes deployment_id, and requires the receipt to mirror the pinned release, runtime image, trust tier, registry reference, ConfigMap references, workload identity, verification policy, and selected workload inventory.

This keeps one environment-specific deployment authority and avoids two nominally equivalent receipts drifting. The fetched release-set.v1 and selected workload pack are verified separately before the ready manifest is published.

Require explicit trust tier and one verified ConfigMap snapshot

trust_tier is mandatory in both the v2 index and its runtime_artifact_delivery block, and the two values must match. Allowed values are production and non_production; environment names do not infer trust.

Production requires a digest-pinned trust_policy_ref and verify.attestations: required_for_prod. The plan expresses a requested minimum. A trusted runtime policy may strengthen it and cannot be weakened by the plan. Production without a configured concrete attestation verifier fails with DPONE_ARTIFACT_ATTESTATION_REQUIRED before registry construction or registry I/O.

ConfigMap names are selectors, not integrity evidence. The init process acquires one bounded snapshot of each selected file, verifies the SHA-256 of those exact bytes, and parses those same bytes. A mismatch fails before registry I/O. Changing configuration requires a new reviewed ConfigMap snapshot and v2 deployment identity; in-place mutation is not a rollout mechanism.

Reserved pod storage is fixed:

Volume	Init mount	Base mount
dpone-fetched-artifacts	RW /var/lib/dpone/artifacts	RO /var/lib/dpone/artifacts
dpone-worktree	RW /workspace/repo	RO /workspace/repo
dpone-run-output	absent	RW /var/lib/dpone/run
dpone-artifact-registry-config	RO /etc/dpone/artifact-registry	absent
dpone-artifact-trust-policy	RO /etc/dpone/artifact-trust when selected	absent

The provider rejects collisions with image, command, arguments, namespace, service account, security context, init containers, reserved volumes, and reserved mounts. Scheduling and resource fields remain the bounded extension surface.

Strict scheduler extensions are carried only by the closed dpone.airflow-provider-execution.v1 projection inside the workload pack. The projection contains bounded task identity/labels/environment plus scheduling fields and the single base-container resource request. It is part of the canonically derived whole-pack fingerprint. Producer, materializer, provider, runtime init and base launcher recompute that identity instead of comparing copied claims. Legacy top-level kpo_kwargs, pod_spec and runtime_command remain available to the explicit raw/local compatibility lane, but the indexed v2 provider never treats them as scheduler authority. A missing projection requires an immutable pack/release/deployment rebuild; unknown or reserved projection fields fail closed. This explicit split replaces the earlier ambiguous behavior where the provider could silently discard legacy executable fields.

The dependency-light provider package is the single structural authority for this projection. Its constants produce the JSON Schema consumed by the root GitOps schema catalog and constrain the executable validator; independently maintained schema copies are forbidden. Structural boundaries are covered by schema/runtime parity tests.

Logical workload identity is not a Kubernetes label contract. The complete ID remains authoritative in the pack, DAG spec, artifact identity and evidence. The shared provider contract derives a Kubernetes-safe label value: an already valid value of at most 63 characters is preserved, while every other value uses a bounded readable prefix plus a SHA-256 suffix. The complete ID is placed in a provider-owned pod annotation. Producer and provider must use this same projection, preventing admission-time failures and prefix-truncation collisions without breaking existing short IDs.

Record the current trusted computing base

For this increment, the trusted computing base includes:

• the producer and immutable release/deployment identity process;
• the promoted local cache snapshot and airflow-index.json;
• the Airflow scheduler/DAG processor and installed dpone provider code;
• the Kubernetes control plane and kubelet enforcing the admitted pod;
• the digest-pinned runtime image and its dpone CLI/runtime code;
• workload-identity/IAM enforcement, the selected registry adapter, and the configured attestation verifier and policy snapshot.

The scheduler, provider, and local cache are explicitly trusted. This decision does not claim resistance to their compromise. A malicious scheduler or provider can submit a different pod or plan, and a malicious trusted cache can change the input accepted by that provider. The fixed KPO contract narrows pack-owned and ordinary override authority inside this TCB.

Consequences

Positive consequences:

• v1 compatibility and v2 executable behavior cannot be confused;
• incomplete or unsupported delivery fails before partial DAG installation;
• the base container cannot use legacy scheduler-copied shell or inline archive fields on the strict path;
• configuration, artifacts, runtime image, and workload identity are bound to one reviewable deployment contract;
• init failure occurs before the base workload, data I/O, XCom success, checkpoint mutation, or source-state advance.

Costs and limitations:

• existing v1 init_fetch indexes must be regenerated, published, materialized, and promoted under a new immutable deployment identity;
• producer, provider, and runtime image versions must understand the same v2 wire;
• production remains fail-closed until a concrete verifier is configured;
• compromised-scheduler resistance would require a separate admission, signing, or policy-enforcement design outside the current TCB.

Evidence and certification status

Current unit and contract tests exercise v1/v2 mode policy, canonical plan transport, fixed pod composition, snapshot digest mismatch, ready-manifest publication, deployment/release receipt verification, post-init checksum and attestation-downgrade rejection, and shell-free argv preparation.

Live Kubernetes projected volumes, workload identity, production attestation, Vault integration, MSSQL, and ClickHouse execution are UNVERIFIED for this decision unless current evidence from the exact release commit and environment is attached. Local or mocked tests are not live certification.

Related

• Approved executable init-fetch feature design
• ADR 0008: Airflow parse-safe provider
• ADR 0009: Artifact delivery and cache materializer
• Provider and cache migration guide
• Airflow cache sync and recovery runbook