Manifest sparse paths

dpone manifest sparse-paths prints the minimal repo-relative allowlist needed to sparse-checkout a manifest and the manifest-owned files it references. Use it whenever a sparse checkout must be generated from dpone manifest metadata instead of hand-maintained scheduler code.

This is a control-plane helper for GitOps runners, Airflow tasks, and KubernetesPodOperator style jobs that should fetch only the files needed for a single workload. The command does not know Airflow internals and does not execute the manifest.

Quickstart

dpone manifest sparse-paths \
  dpone_workloads/manifests/mssql/orders.yaml \
  --include-global-overrides \
  --include-env-overrides dev \
  --include-registry \
  --support-path dpone_workloads/sql/orders/

Sparse output is the default:

dpone_workloads/manifests/mssql/orders.yaml
dpone_workloads/overrides/global.yaml
dpone_workloads/overrides/dev.yaml
dpone_workloads/registry/
dpone_workloads/sql/orders/

Feed it directly to Git:

dpone manifest sparse-paths dpone_workloads/manifests/mssql/orders.yaml \
  --include-env-overrides dev \
  --include-registry \
  | git sparse-checkout set --stdin

Use JSON for CI logs, documentation examples, and debugging:

dpone manifest sparse-paths dpone_workloads/manifests/mssql/orders.yaml \
  --include-env-overrides dev \
  --format json

What gets included

The root manifest is always included first. dpone then reads raw YAML and adds manifest-owned dependencies without compiling the manifest or loading runtime connectors:

• custom convention and conventions YAML files;
• registry and registries YAML files;
• file-backed depends_on.path references, including file.yaml#selector;
• recursive manifest dependencies with cycle protection.

Local selector dependencies such as #public.orders stay inside the current manifest and do not add a new sparse path.

Group dependencies such as depends_on: [{group: landing_shared}] do not encode a file path. The JSON output records a warning so the GitOps wrapper can decide whether to include a broader path or add an explicit support path.

Path safety

All public output is repo-relative. The validator rejects absolute paths, empty paths, .., and paths outside the workload root. This safety rule belongs to dpone, not to Airflow or GitHub Actions wrappers.

When --workload-root is omitted, dpone infers it from the nearest manifests/ segment. For dpone_workloads/manifests/mssql/orders.yaml, the workload root is dpone_workloads.

Optional requested paths are deterministic. If --include-env-overrides dev is used and dpone_workloads/overrides/dev.yaml does not exist yet, sparse output still includes it and JSON marks exists=false with a warning.

Runbook

1. Run dpone manifest sparse-paths <manifest> --format json in CI and inspect warnings before wiring a new runner.
2. Add --include-global-overrides, --include-env-overrides <env>, and --include-registry only when your workload convention uses those files.
3. Add explicit --support-path entries for SQL, templates, or other assets until those assets have first-class manifest dependency fields.
4. Store the sparse output as a job artifact when debugging GitOps checkout differences.
5. Keep sparse-checkout allowlists generated by dpone, not duplicated in Airflow DAG code.

Common fixes

Symptom	Fix
invalid_path blocker	Remove absolute paths, .., or paths outside the workload root.
Missing override warning	Create the override file or keep the deterministic future path.
Group dependency warning	Add an explicit depends_on.path or a broader --support-path.
SQL/template file missing in runner	Add --support-path until a manifest dependency field exists.