Skip to content

CI/CD

dpone uses GitHub Actions as the public OSS automation path. The default branch for the public release flow is master.

This page is the CI/CD index. It explains the automation model at a high level and links to detailed workflow docs, failure runbooks, and developer guidance for changing CI/CD safely.

CI/CD map

flowchart LR
    Dev["Developer branch or fork"] --> PR["Pull request to master"]
    PR --> CI["CI quality matrix"]
    PR --> Docs["Docs strict build"]
    PR --> Security["Secret scan and CodeQL"]
    CI --> Review["Review and merge"]
    Docs --> Review
    Security --> Review
    Review --> Master["master"]
    Master --> Pages["GitHub Pages deploy"]
    Master --> Scorecard["OSSF Scorecard"]
    Master --> Manual["Manual integration matrix"]
    Master --> OssQuality["OSS code quality benchmark"]
    Manual --> Obs["Observability maturity"]
    Tag["vX.Y.Z tag"] --> Release["Build package and publish to PyPI"]
    Manual --> Cert["Connector certification artifacts"]
    Manual --> Route["Route readiness artifacts"]
    Route --> CdcApply["CDC apply certification artifacts"]
    CdcApply --> Cdc["CDC handoff artifacts"]
    Cdc --> CdcObs["CDC observability evidence"]
    CdcObs --> CdcRecovery["CDC recovery evidence"]
    CdcRecovery --> CdcSchema["CDC schema evolution evidence"]
    CdcSchema --> CdcSchemaApply["CDC schema apply"]
    CdcSchemaApply --> CdcPromotion["CDC promotion gate"]
    CdcPromotion --> CdcRuntime["CDC runtime run"]
    Cert --> Maturity["Production maturity gate"]
    CdcPromotion --> Industrial
    CdcRuntime --> CdcPoison["CDC poison quarantine and replay"]
    CdcPoison --> CdcCompare["CDC compare and repair"]
    CdcCompare --> Industrial
    Obs --> Maturity
    Maturity --> Industrial["Industrial readiness gate"]
    Industrial --> Release
    Maturity --> Release

Detailed docs

Need Doc
Understand each workflow, trigger, job, artifact, and local reproduction command Workflow reference
Recover a red CI/CD run Failure runbooks
Understand release publishing, GitHub Pages, and PyPI Trusted Publishing Release and Pages automation
Run the manual source/sink matrix Manual integration matrix and matrix runbook
Certify connectors Connector certification
Add or change CI/CD as a developer Developer CI/CD guide
Run the local/default test gates Testing and Testing runbooks
Prepare Airflow Kubernetes runners for custom dpone images GitOps Airflow runner pack

Workflow inventory

Workflow File Trigger Required by default Purpose
CI .github/workflows/ci.yml push to master, pull request to master, manual dispatch yes Lint, format check, type check, non-live tests, coverage XML, package build, PostgreSQL XMin integration.
docs .github/workflows/pages.yml docs changes on push/PR, manual dispatch yes for docs changes Generated-reference check plus strict MkDocs build; deploys GitHub Pages from master.
Secret Scan .github/workflows/secret-scan.yml push, PR, weekly schedule yes TruffleHog verified secret scan.
CodeQL .github/workflows/codeql.yml push, PR, weekly schedule yes Python static security analysis.
OSSF Scorecard .github/workflows/scorecard.yml push to master, branch protection changes, weekly schedule advisory/security Supply-chain posture analysis and SARIF upload.
Release .github/workflows/release.yml vX.Y.Z tag, manual dispatch release-only Build sdist/wheel, run twine check, publish to PyPI through Trusted Publishing or token fallback.
OSS code quality benchmark .github/workflows/oss-code-quality-benchmark.yml manual dispatch manual/docs Refreshes dpone vs OSS comparator LOC/SLOC, coupling, cohesion, SOLID, Clean OOP, test-footprint, Complexity & Boundary Discipline, Semantic Maintainability Deep Scan, Scoring Validity & Calibration, Scale Readiness & Growth Simulation, Refactor ROI Roadmap, quality debt, Evidence Trust & Auditability, Public Evidence Integrity, Source Citation Verification, Benchmark v3 Release Readiness, Claims Ledger, Runtime Certification Matrix, Executable Certification, Quality Budget As Code, Evidence Warehouse Export, release readiness pack, claim-to-source matrix, Independent Analyzer Cross-Validation, External analyzer execution with tokei/cloc/radon/lizard, public artifact redaction, claim coverage, external analyzer command ledger, stale analyzer values, stale source values, freshness metadata, historical trend evidence, PR regression gate, PR summary, and benchmark visuals; opens a PR with generated docs when requested.
Source/Sink integration matrix .github/workflows/integration-matrix.yml manual dispatch manual Runs all source -> sink strategy contracts or local/mock matrix layers with artifacts.
Backfill integration matrix .github/workflows/backfill-integration.yml PR (path-filtered on backfill/runtime/sink/source code), nightly schedule, manual dispatch path-triggered PR + nightly End-to-end chunked backfill certification against disposable Postgres/ClickHouse/MSSQL/Kafka services: route x inner-strategy matrix, resume/parallel/verification scenarios, interval-driven Airflow runs; uploads junit and coverage evidence. See testing/backfill-integration.md.
Airflow provider compatibility .github/workflows/airflow-pack-compat.yml PR/push path-filtered on the provider, reader and interval contract, manual dispatch path-triggered PR Validates apache-airflow-providers-dpone plus dpone-airflow-pack against pinned Airflow 2.10.5, 2.11.0, 3.2.0, and 3.3.0 with exact CNCF Kubernetes provider pins: discovery/import safety, cache IO, and real-Airflow Jinja rendering of the DPONE_* interval env templates.
Observability maturity .github/workflows/observability-maturity.yml manual dispatch, weekly schedule scheduled/manual Exports Prometheus/OpenTelemetry metrics, validates SLO smoke, and publishes observability evidence.
Connector certification .github/workflows/connector-certification.yml daily schedule, manual dispatch scheduled/manual Async scheduled release-evidence gate with offline certification, Docker local-live connector markers, and vendor/API live tests.
Production maturity .github/workflows/production-maturity.yml weekly schedule, manual dispatch scheduled/manual Aggregates certification, CDC, performance, security, supply-chain, governance, and docs evidence into one release readiness report.
Industrial readiness .github/workflows/industrial-readiness.yml weekly schedule, manual dispatch scheduled/manual Aggregates local matrix, correctness, reliability, performance lab, UX, and governance evidence into one industrial readiness report.
Route certification release .github/workflows/route-certification-release.yml manual dispatch manual/release Runs dpone ops route-certify-release over first-class route bundles and uploads route_certification_release.json, Markdown, and release notes.
Route release finalize .github/workflows/route-release-finalize.yml manual dispatch manual/release Runs dpone ops route-release-finalize with bundle discovery, freshness/provenance/regression checks, and history artifacts.

Default PR gate

For normal code and documentation changes, contributors should be able to reproduce the default gate locally:

uv sync --all-extras
uv run ruff check .
uv run ruff format --check .
uv run mypy --config-file mypy.ini
uv run dpone docs check-module-size --package packages/dpone-airflow-pack/src/dpone_airflow_pack --no-baseline
uv run dpone docs check-module-size --package packages/apache-airflow-providers-dpone/src/airflow/providers/dpone --no-baseline
uv run dpone docs check-generated-references
uv run dpone docs check-airflow-public-contracts
uv run dpone docs update-airflow-public-contract-reference --check
uv run pytest -m "not integration_live" -n auto --cov=src/dpone --cov=packages/dpone-airflow-pack/src/dpone_airflow_pack --cov-report=xml
uv build
uv build packages/apache-airflow-providers-dpone --out-dir dist

The unit/contract layer runs in parallel via pytest-xdist (-n auto); import-state-sensitive guards isolate themselves in fresh subprocesses so the suite stays order-independent. The coverage gate (fail_under in [tool.coverage.report]) is enforced by pytest-cov and follows a ratchet policy: raise it toward the measured baseline, never lower it.

For docs changes, also run:

python -m pip install -r docs/requirements.txt
mkdocs build --strict

For release candidates, add:

uv run twine check dist/*

Vault-backed CI secrets

CI jobs that need HashiCorp Vault-compatible KV access must install the vault extra or the public vault-kv-client package and import only vault_kv_client. The dpone runtime, CLI, and CI acceptance helpers must not use legacy Vault import paths, even if a third-party package keeps them as a compatibility alias.

GitHub/GitLab job tokens, Vault ID tokens, Airflow service-account passwords, and kubeconfig payloads must stay in the CI secret store or Vault. They must not be committed to repository files, rendered into Markdown artifacts, or printed in logs.

Schema migration promotion gate

Repository systems remain the canonical approval layer. In GitHub, GitLab, Bitbucket, or any equivalent PR/MR flow, use dpone migration promotion artifacts as CI evidence rather than rebuilding packs per environment.

Suggested pipeline:

uv run dpone schema migration plan \
  --manifest manifests/orders.yaml \
  --actual actual/stage-clickhouse-physical.json \
  --format json \
  --output .dpone/schema-migration/orders.pack.json

uv run dpone schema impact plan \
  --pack .dpone/schema-migration/orders.pack.json \
  --manifest manifests/orders.yaml \
  --format json \
  --output .dpone/schema-impact/orders.impact.json

uv run dpone schema impact gate \
  --pack .dpone/schema-migration/orders.pack.json \
  --impact .dpone/schema-impact/orders.impact.json \
  --approval approvals/orders-impact.yaml

uv run dpone schema contract consumers lineage \
  --manifest manifests/orders.yaml \
  --format json \
  --output .dpone/schema-contracts/orders.lineage.json

uv run dpone schema contract consumers discover \
  --manifest manifests/orders.yaml \
  --lineage .dpone/schema-contracts/orders.lineage.json \
  --format json \
  --output .dpone/schema-contracts/orders.consumers.json

uv run dpone schema contract consumers matrix \
  --manifest manifests/orders.yaml \
  --against analytics.orders@1.5.0 \
  --consumers .dpone/schema-contracts/orders.consumers.json \
  --lineage .dpone/schema-contracts/orders.lineage.json \
  --format json \
  --output .dpone/schema-contracts/orders.consumer-matrix.json

uv run dpone schema contract consumers gate \
  --manifest manifests/orders.yaml \
  --pack .dpone/schema-migration/orders.pack.json \
  --matrix .dpone/schema-contracts/orders.consumer-matrix.json \
  --format json \
  --output .dpone/schema-contracts/orders.consumer-gate.json

uv run dpone schema contract views plan \
  --manifest manifests/orders.yaml \
  --against analytics.orders@1.5.0 \
  --consumer-matrix .dpone/schema-contracts/orders.consumer-matrix.json \
  --format json \
  --output .dpone/schema-contracts/orders.compatibility-views.json

uv run dpone schema contract views gate \
  --plan .dpone/schema-contracts/orders.compatibility-views.json \
  --consumer-gate .dpone/schema-contracts/orders.consumer-gate.json \
  --format json \
  --output .dpone/schema-contracts/orders.compatibility-view-gate.json

uv run dpone schema contract consumers test-kit plan \
  --manifest manifests/orders.yaml \
  --matrix .dpone/schema-contracts/orders.consumer-matrix.json \
  --compatibility-view-plan .dpone/schema-contracts/orders.compatibility-views.json \
  --format json \
  --output .dpone/schema-contracts/orders.consumer-test-kit.json

uv run dpone schema contract consumers test-kit render \
  --kit .dpone/schema-contracts/orders.consumer-test-kit.json \
  --format pytest \
  --output tests/generated/test_orders_consumers.py

uv run dpone schema contract consumers test-kit certify \
  --kit .dpone/schema-contracts/orders.consumer-test-kit.json \
  --result passed \
  --format json \
  --output .dpone/schema-contracts/orders.consumer-certification.json

uv run dpone schema contract adoption plan \
  --manifest manifests/orders.yaml \
  --consumer-matrix .dpone/schema-contracts/orders.consumer-matrix.json \
  --compatibility-view-plan .dpone/schema-contracts/orders.compatibility-views.json \
  --format json \
  --output .dpone/schema-contracts/orders.adoption-plan.json

uv run dpone schema contract adoption status \
  --plan .dpone/schema-contracts/orders.adoption-plan.json \
  --registry .dpone/schema-migration/registry/registry.json \
  --consumer-certification .dpone/schema-contracts/orders.consumer-certification.json \
  --format json \
  --output .dpone/schema-contracts/orders.adoption-status.json

uv run dpone schema contract adoption gate \
  --status .dpone/schema-contracts/orders.adoption-status.json \
  --profile prod_strict \
  --format json \
  --output .dpone/schema-contracts/orders.retirement-gate.json

Then build one attested review bundle and render Markdown for the PR/MR:

uv run dpone schema migration bundle build \
  --pack .dpone/schema-migration/orders.pack.json \
  --impact .dpone/schema-impact/orders.impact.json \
  --consumer-gate .dpone/schema-contracts/orders.consumer-gate.json \
  --consumer-certification .dpone/schema-contracts/orders.consumer-certification.json \
  --compatibility-view-gate .dpone/schema-contracts/orders.compatibility-view-gate.json \
  --contract-adoption-status .dpone/schema-contracts/orders.adoption-status.json \
  --contract-retirement-gate .dpone/schema-contracts/orders.retirement-gate.json \
  --environment-contract .dpone/environments/orders.yaml \
  --approval approvals/orders-impact.yaml \
  --attest \
  --output-dir .dpone/schema-migration/review/orders \
  --format json

uv run dpone schema migration bundle verify \
  --bundle .dpone/schema-migration/review/orders/bundle.json \
  --require-attestation \
  --format json \
  --output .dpone/schema-migration/review/orders/verification.json

uv run dpone schema migration bundle attest \
  --bundle .dpone/schema-migration/review/orders/bundle.json \
  --output .dpone/schema-migration/review/orders/provenance.json \
  --signing-key-env DPONE_BUNDLE_SIGNING_KEY \
  --signing-key-id ci-hmac \
  --provenance-source github_actions \
  --repository "$GITHUB_SERVER_URL/$GITHUB_REPOSITORY" \
  --commit-sha "$GITHUB_SHA" \
  --ref "$GITHUB_REF" \
  --run-id "$GITHUB_RUN_ID" \
  --workflow ".github/workflows/schema-migration.yml" \
  --protected-ref \
  --format json

uv run dpone schema migration bundle trust verify \
  --bundle .dpone/schema-migration/review/orders/bundle.json \
  --provenance .dpone/schema-migration/review/orders/provenance.json \
  --signing-key-env DPONE_BUNDLE_SIGNING_KEY \
  --format json \
  --output .dpone/schema-migration/review/orders/trust-verification.json

if [ -f .dpone/connections/clickhouse-stage.json ]; then
  uv run dpone schema migration rehearse plan \
    --pack .dpone/schema-migration/orders.pack.json \
    --bundle .dpone/schema-migration/review/orders/bundle.json \
    --environment stage \
    --target-connection .dpone/connections/clickhouse-stage.json \
    --output-dir .dpone/schema-migration/rehearsal/orders \
    --format json
  uv run dpone schema migration rehearse fixture plan \
    --pack .dpone/schema-migration/orders.pack.json \
    --manifest manifests/orders.yaml \
    --format json \
    --output .dpone/schema-migration/rehearsal/orders/fixture-plan.json
  uv run dpone schema migration rehearse fixture build \
    --plan .dpone/schema-migration/rehearsal/orders/fixture-plan.json \
    --target-connection .dpone/connections/clickhouse-stage.json \
    --execute \
    --format json \
    --output .dpone/schema-migration/rehearsal/orders/fixture-build.json
  uv run dpone schema migration rehearse fixture profile \
    --fixture-build .dpone/schema-migration/rehearsal/orders/fixture-build.json \
    --target-connection .dpone/connections/clickhouse-stage.json \
    --stage before \
    --format json \
    --output .dpone/schema-migration/rehearsal/orders/profile-before.json
  uv run dpone schema migration rehearse run \
    --plan .dpone/schema-migration/rehearsal/orders/rehearsal-plan.json \
    --execute \
    --format json \
    --output .dpone/schema-migration/rehearsal/orders/rehearsal-run.json
  uv run dpone schema migration rehearse fixture profile \
    --fixture-build .dpone/schema-migration/rehearsal/orders/fixture-build.json \
    --target-connection .dpone/connections/clickhouse-stage.json \
    --stage after \
    --format json \
    --output .dpone/schema-migration/rehearsal/orders/profile-after.json
  uv run dpone schema migration rehearse certify \
    --run .dpone/schema-migration/rehearsal/orders/rehearsal-run.json \
    --fixture-build .dpone/schema-migration/rehearsal/orders/fixture-build.json \
    --before-profile .dpone/schema-migration/rehearsal/orders/profile-before.json \
    --after-profile .dpone/schema-migration/rehearsal/orders/profile-after.json \
    --profile prod_strict \
    --format json \
    --output .dpone/schema-migration/rehearsal/orders/rehearsal-certificate.json
fi

uv run dpone schema migration bundle gate \
  --bundle .dpone/schema-migration/review/orders/bundle.json \
  --profile pr_review \
  --trust-verification .dpone/schema-migration/review/orders/trust-verification.json \
  --format json \
  --output .dpone/schema-migration/review/orders/bundle-gate.json

uv run dpone schema migration registry record \
  --bundle .dpone/schema-migration/review/orders/bundle.json \
  --gate .dpone/schema-migration/review/orders/bundle-gate.json \
  --trust .dpone/schema-migration/review/orders/trust-verification.json \
  --environment pr \
  --stage reviewed \
  --format json \
  --output .dpone/schema-migration/review/orders/evidence-record.json

uv run dpone schema migration bundle diff \
  --base .dpone/schema-migration/base/orders/bundle.json \
  --head .dpone/schema-migration/review/orders/bundle.json \
  --head-gate .dpone/schema-migration/review/orders/bundle-gate.json \
  --require-attestation \
  --format md \
  --output .dpone/schema-migration/review/orders/diff.md

uv run dpone schema migration review render \
  --bundle .dpone/schema-migration/review/orders/bundle.json \
  --format md \
  --output .dpone/schema-migration/review/orders/review.md

SCM owns branch protection, PR/MR approvals, protected environments, and deploy job authorization. dpone owns pack consistency, artifact SHA-256 integrity, bundle_id, bundle_digest, schema impact evidence, bundle gate policy receipts, bundle diff review deltas, trusted provenance receipts, environment promotion receipts, evidence registry records, and fail-closed apply gates. Provider-specific comment publishers are optional wrappers over review.md; the core bundle contract is SCM-neutral.

Credential-free templates are available under:

  • docs/examples/ci/schema-migration/github-actions.yml
  • docs/examples/ci/schema-migration/gitlab-ci.yml
  • docs/examples/ci/schema-migration/bitbucket-pipelines.yml

After the stage deploy job applies the pack, certify the exact pack_id:

uv run dpone schema migration verify-env \
  --pack .dpone/schema-migration/orders.pack.json \
  --environment stage \
  --environment-contract .dpone/environments/orders.yaml \
  --format json

uv run dpone schema migration certify \
  --pack .dpone/schema-migration/orders.pack.json \
  --environment stage \
  --environment-contract .dpone/environments/orders.yaml \
  --output .dpone/schema-migration/orders.stage.cert.json \
  --format json

The prod deploy job must consume the certification plus a repository-approved promotion approval artifact:

uv run dpone schema migration promote \
  --pack .dpone/schema-migration/orders.pack.json \
  --from stage \
  --to prod \
  --certificate .dpone/schema-migration/orders.stage.cert.json \
  --environment-contract .dpone/environments/orders.yaml \
  --approval approvals/orders-prod.yaml \
  --output .dpone/schema-migration/orders.stage-to-prod.promotion.json \
  --format json

uv run dpone schema migration apply \
  --plan .dpone/schema-migration/orders.pack.json \
  --environment prod \
  --environment-contract .dpone/environments/orders.yaml \
  --promotion .dpone/schema-migration/orders.stage-to-prod.promotion.json \
  --execute \
  --format json

uv run dpone schema migration post-apply plan \
  --pack .dpone/schema-migration/orders.pack.json \
  --bundle .dpone/schema-migration/review/orders/bundle.json \
  --ledger .dpone/schema-migration/ledger.json \
  --manifest manifests/orders.yaml \
  --target-connection .dpone/connections/clickhouse-prod.json \
  --environment prod \
  --format json \
  --output .dpone/schema-migration/post-apply/orders/post-apply-plan.json

uv run dpone schema migration post-apply run \
  --plan .dpone/schema-migration/post-apply/orders/post-apply-plan.json \
  --execute \
  --format json \
  --output .dpone/schema-migration/post-apply/orders/post-apply-run.json

uv run dpone schema migration post-apply certify \
  --run .dpone/schema-migration/post-apply/orders/post-apply-run.json \
  --profile prod_strict \
  --format json \
  --output .dpone/schema-migration/post-apply/orders/post-apply-certificate.json

uv run dpone schema migration watch plan \
  --pack .dpone/schema-migration/orders.pack.json \
  --post-apply-certificate .dpone/schema-migration/post-apply/orders/post-apply-certificate.json \
  --manifest manifests/orders.yaml \
  --target-connection .dpone/connections/clickhouse-prod.json \
  --environment prod \
  --format json \
  --output .dpone/schema-migration/watch/orders/watch-plan.json

uv run dpone schema migration watch run \
  --plan .dpone/schema-migration/watch/orders/watch-plan.json \
  --execute \
  --format json \
  --output .dpone/schema-migration/watch/orders/watch-run.json

uv run dpone schema migration watch certify \
  --run .dpone/schema-migration/watch/orders/watch-run.json \
  --profile prod_strict \
  --format json \
  --output .dpone/schema-migration/watch/orders/watch-certificate.json

uv run dpone data product assertions plan \
  --manifest manifests/orders.yaml \
  --format json \
  --output .dpone/data-products/orders.assertion-plan.json

uv run dpone data product assertions evaluate \
  --plan .dpone/data-products/orders.assertion-plan.json \
  --runtime-artifact .dpone/runs/orders/latest-run.json \
  --target-connection .dpone/connections/clickhouse-prod.json \
  --format json \
  --output .dpone/data-products/orders.assertion-evaluation.json

uv run dpone data product assertions gate \
  --evaluation .dpone/data-products/orders.assertion-evaluation.json \
  --profile prod_strict \
  --format json \
  --output .dpone/data-products/orders.assertion-gate.json

uv run dpone data product slo plan \
  --manifest manifests/orders.yaml \
  --contract-gate .dpone/schema-contracts/orders.contract-gate.json \
  --consumer-gate .dpone/schema-contracts/orders.consumer-gate.json \
  --assertion-gate .dpone/data-products/orders.assertion-gate.json \
  --format json \
  --output .dpone/data-products/orders.slo-plan.json

uv run dpone data product slo evaluate \
  --plan .dpone/data-products/orders.slo-plan.json \
  --registry .dpone/schema-migration/registry.sqlite3 \
  --runtime-artifact .dpone/runs/orders/latest-run.json \
  --target-connection .dpone/connections/clickhouse-prod.json \
  --format json \
  --output .dpone/data-products/orders.slo-evaluation.json

uv run dpone data product slo gate \
  --evaluation .dpone/data-products/orders.slo-evaluation.json \
  --profile prod_strict \
  --format json \
  --output .dpone/data-products/orders.slo-gate.json

uv run dpone data product incident report \
  --evaluation .dpone/data-products/orders.slo-evaluation.json \
  --slo-gate .dpone/data-products/orders.slo-gate.json \
  --format md \
  --output .dpone/data-products/orders.incident.md

# Run this branch only when the watch certificate recommends or requires rollback.
uv run dpone schema migration remediation plan \
  --pack .dpone/schema-migration/orders.pack.json \
  --watch-certificate .dpone/schema-migration/watch/orders/watch-certificate.json \
  --ledger .dpone/schema-migration/ledger.json \
  --manifest manifests/orders.yaml \
  --target-connection .dpone/connections/clickhouse-prod.json \
  --environment prod \
  --format json \
  --output .dpone/schema-migration/remediation/orders/remediation-plan.json

uv run dpone schema migration remediation apply \
  --plan .dpone/schema-migration/remediation/orders/remediation-plan.json \
  --approval approvals/orders-rollback.yaml \
  --execute \
  --format json \
  --output .dpone/schema-migration/remediation/orders/remediation-run.json

uv run dpone schema migration remediation certify \
  --run .dpone/schema-migration/remediation/orders/remediation-run.json \
  --target-connection .dpone/connections/clickhouse-prod.json \
  --profile prod_strict \
  --format json \
  --output .dpone/schema-migration/remediation/orders/remediation-certificate.json

uv run dpone schema migration registry record \
  --bundle .dpone/schema-migration/review/orders/bundle.json \
  --environment prod \
  --stage verified \
  --post-apply-certificate .dpone/schema-migration/post-apply/orders/post-apply-certificate.json \
  --format json

uv run dpone schema migration registry record \
  --bundle .dpone/schema-migration/review/orders/bundle.json \
  --environment prod \
  --stage watched \
  --watch-certificate .dpone/schema-migration/watch/orders/watch-certificate.json \
  --format json

uv run dpone schema migration registry record \
  --bundle .dpone/schema-migration/review/orders/bundle.json \
  --environment prod \
  --stage slo_gate_passed \
  --data-product-slo-gate .dpone/data-products/orders.slo-gate.json \
  --format json

uv run dpone schema migration registry record \
  --bundle .dpone/schema-migration/review/orders/bundle.json \
  --environment prod \
  --stage remediated \
  --remediation-certificate .dpone/schema-migration/remediation/orders/remediation-certificate.json \
  --format json

This keeps branch switching and PR/MR approvals simple: SCM validates who may merge or deploy; dpone validates that prod applies the same pack certified in stage, proves the applied target state through read-only ledger, physical drift, data profile, canary, and rollback-window checks, then watches the target window with canary/query-health evidence. If watch requires rollback, dpone executes only a controlled, approved remediation plan and records the remediation certificate before closing the release.

GitHub Actions runtime baseline

All first-party and core third-party workflow actions should stay on Node 24-compatible major versions. This prevents GitHub Actions deprecation warnings from leaking into release, Pages, and certification gates.

Current baseline:

Action family Minimum major used by dpone Reason
actions/checkout v6 Node 24 action runtime.
actions/setup-python v6 Node 24 action runtime.
astral-sh/setup-uv v7 Node 24 action runtime.
actions/upload-artifact v6 Node 24 action runtime.
actions/upload-pages-artifact v5 Pages artifact upload on the current runtime line.
actions/configure-pages v6 Pages setup on the current runtime line.
actions/deploy-pages v5 Pages deploy on the current runtime line.
github/codeql-action/* v4 Supported CodeQL line with current runtime support.
softprops/action-gh-release v3 Node 24 action runtime for GitHub Releases.

Self-hosted runners must be new enough to support Node 24 actions. If a workflow fails immediately during action startup, update the runner first, then re-run the workflow before changing dpone code.

Every astral-sh/setup-uv step must also set version: "0.11.28". The workflow governance tests fail when a new workflow installs uv from the action default or from the latest available release. Update the pinned action SHA and the uv runtime version together in a reviewed maintenance PR.

Manual and scheduled gates

Manual/scheduled gates are intentionally separate from ordinary PR CI because they may start services, use external credentials, or run longer certification suites.

Gate When to run Command or workflow
Source/sink matrix mock_contract Docs/manifest/strategy support changes GitHub Actions Source/Sink integration matrix with run_mode=mock_contract.
Source/sink matrix mock_local Connector or staging behavior changes GitHub Actions Source/Sink integration matrix with run_mode=mock_local.
Real-local release certification Minor/major release candidates GitHub Actions Live certification with profile=real_local; requires green release-evidence-pack.
Type matrix certification Type mapping, schema evolution, physical DDL, temporal fidelity or critical-route native transfer changes GitHub Actions Live certification with profile=type_matrix_certification; validates contract suite plus local Docker focused route fixtures.
Native transfer certification Critical native transfer route changes GitHub Actions Live certification with profile=native_transfer; builds native evidence, strategy certification bundle, and native-transfer release evidence pack.
Vendor live certification Release candidates and managed connector changes Connector certification with run_vendor_live=true; runs provider/API directories only and must not pick up local Docker route tests.
Connector local-live certification Connector runtime changes Scheduled/manual Connector certification; installs ODBC Driver 18 plus mssql-tools18, starts Postgres, MSSQL, ClickHouse, Kafka, Schema Registry, and MinIO, and materializes the MSSQL test database before marker tests.
Observability maturity Metrics, SLO, run-report, or artifact evidence changes GitHub Actions Observability maturity.
Production maturity Release candidates and weekly readiness review GitHub Actions Production maturity or uv run dpone ops production-maturity.
Industrial readiness Industrial release candidates and readiness review GitHub Actions Industrial readiness or uv run dpone ops industrial-readiness.
OSS code quality benchmark Public quality benchmark refresh after major architecture/docs/runtime changes GitHub Actions OSS code quality benchmark; use project=all for a full refresh or a single project to recover a stale comparator.
Route readiness Critical source -> sink route changes after evidence generation uv run dpone ops route-certification-pack, then review embedded route_readiness.json.
Route execution ledger Retry/resume, state commit, fencing, CDC apply, repair, or resync changes for any route uv run dpone ops route-execution-ledger; use --store-backend sqlite for shared-runner certification, then attach route_execution_ledger.json to route readiness and release evidence.
Route state promotion Source-state, offset, xmin, LSN, cursor, or checkpoint promotion changes for any route uv run dpone ops route-state-promote; see Route state promotion, use --state-backend sqlite for shared-runner certification, then attach state_promotion.json to route readiness and release evidence.
Route live certification Docker-live/vendor-live route release candidates after live evidence exists uv run dpone ops route-live-certification; see Route live certification, upload route_live_certification.json, and require it in route-release-gate as route_live_evidence_bundle.
Route certify Minor/major route release candidates after refresh execute, snapshot capture, exact verify, readiness, and release evidence exist uv run dpone ops route-certify; see Route certify, upload route_certification_bundle.json, and block release tags unless the bundle level is certified.
Route certify release Minor/major release candidates after first-class route bundles exist uv run dpone ops route-certify-release --release vX.Y.Z-rcN --profile oss_ci; use .github/workflows/route-certification-release.yml, upload route_certification_release.json, and block release tags unless level is release_ready.
Route release finalize Final route-certified release candidate review before tagging uv run dpone ops route-release-finalize --release vX.Y.Z-rcN --bundle-root test_artifacts/route_certify; upload route_release_finalizer.json and keep release tags blocked unless level is final_ready.
Route release candidate orchestrator Release candidates that need one reproducible route release train uv run dpone ops route-rc-orchestrator; see Route release candidate orchestrator, upload route_rc_orchestration.json, and treat its release_evidence_pack output as the route RC evidence pack.
Route release candidate executor Route RC receipts that must be dry-run checked or explicitly executed against Docker-live services uv run dpone ops route-rc-execute; see Route release candidate executor, upload route_rc_execution.json, and use --execute only in opt-in local-live/vendor-live gates.
Release RC collector Stacked release-candidate PR trains before final review uv run dpone ops release-rc-collect; see Release RC collector, upload release_rc_collect.json, release_rc_inputs.json, and merge_train.json, then feed the generated command to the finalizer.
Release RC finalizer Final stacked release-candidate review before creating the tag uv run dpone ops release-rc-finalize; see Release RC finalizer, upload release_rc_finalizer.json, and block tags unless level is rc_ready.
Route release gate Route release candidates after readiness, certification pack, execution ledger, state promotion, and optional CDC evidence are available uv run dpone ops route-release-gate; see Route release gate, then attach route_release_gate.json and route_release_gate.md to release review.
Route run supervisor Manifest, route RC, CDC apply, repair, or resync changes that need one lifecycle receipt uv run dpone ops route-run-supervisor; see Route run supervisor, upload route_run_receipt.json, and require it in route-release-gate as route_run_supervisor for critical route release candidates.
Route data quality Data contract, quarantine, reconciliation, or exception-management changes for any critical route uv run dpone ops route-data-quality; see Route data quality, upload route_data_quality.json, and require it in route-release-gate as route_data_quality for release candidates.
Route refresh plan Backfill, replay, repair, retention-gap recovery, schema backfill, or source-state rewind planning for any critical route uv run dpone ops route-refresh-plan; see Route refresh plan, upload route_refresh_plan.json, and require it in route-release-gate as route_refresh_plan when a release executes refresh work.
Route refresh execute, capture, and verify Dry-run or execution receipt for a reviewed route refresh plan, read-only source/sink snapshot capture, followed by post-load source/sink reconciliation uv run dpone ops route-refresh-execute, uv run dpone ops route-refresh-capture-snapshots, and uv run dpone ops route-refresh-verify; see Route refresh execute, upload route_refresh_execution.json, route_refresh_snapshot_capture.json, source_route_refresh_snapshot.json, sink_route_refresh_snapshot.json, and route_refresh_verification.json, and require them in route release evidence when chunks are applied. Keep default OSS CI credential-free; run --executor mssql_clickhouse --executor-config-json ... or --executor postgres_mssql --executor-config-json ... only in opt-in Docker-live/vendor-live jobs with required databases and native tools. Non-live coverage lives in test_route_refresh_snapshot_capture.py, test_cli_route_refresh_snapshot_capture_command.py, test_route_refresh_snapshot_capture_registry.py, test_route_refresh_verify.py, and test_cli_route_refresh_verify_command.py. The opt-in live gate executes 10,000 rows and 200 columns per first-class route, captures snapshots, verifies exact typed hashes, and repeats capture/verify after schema evolution.

Set DPONE_RUN_REFRESH_EXECUTOR_LIVE=1 in the Live certification workflow to enable the route refresh executor live tests. The workflow copies replay receipts to test_artifacts/live_certification/refresh-executor/mssql-clickhouse/route_refresh_execution.json and test_artifacts/live_certification/refresh-executor/postgres-mssql/route_refresh_execution.json, then copies the matching route_refresh_snapshot_capture.json, source_route_refresh_snapshot.json, sink_route_refresh_snapshot.json, and route_refresh_verification.json files beside each replay receipt. | CDC apply certification | CDC apply correctness, delete semantics, typed hash, and replay idempotency changes | uv run dpone ops cdc-apply-certification, then review embedded handoff/cdc_handoff.json. | | CDC snapshot handoff | CDC source -> sink promotion after snapshot and apply evidence generation | uv run dpone ops cdc-handoff, then review cdc_handoff.json. | | CDC observability evidence | CDC lag, freshness, retention, offset, replay, or throughput SLO changes | uv run dpone ops cdc-observability-evidence, then review cdc_observability.json. | | CDC recovery evidence | CDC restart, partial commit, offset ordering, replay, poison event, or retention recovery changes | uv run dpone ops cdc-recovery-evidence, then review cdc_recovery_evidence.json. | | CDC schema evolution evidence | CDC source schema changes, target DDL governance, backfill, breaking-change approval, or offset/schema ordering changes | uv run dpone ops cdc-schema-evolution-evidence, then review cdc_schema_evolution_evidence.json. | | CDC schema apply | CDC target DDL apply, additive column backfill, or typed serving refresh changes | uv run dpone ops cdc-schema-apply --mode dry_run, then approved --mode apply; review cdc_schema_apply_plan.json, cdc_schema_apply_result.json, and optional typed_refresh/cdc_typed_materialization.json. | | CDC promotion gate | Final CDC replication readiness and offset promotion review after all upstream CDC gates are green | uv run dpone ops cdc-promotion-gate, then review cdc_promotion_gate.json. | | CDC runtime run | Bounded CDC read, sink apply, and durable offset commit smoke for replication-grade routes | uv run dpone ops cdc-runtime-run, then review cdc_runtime_run.json. | | CDC poison quarantine and replay | Poison-event classification, quarantine inspection, replay execution, and ClickHouse duplicate replay safety | uv run dpone ops cdc-runtime-run --poison-mode quarantine_and_continue, then uv run dpone ops cdc-quarantine-inspect and uv run dpone ops cdc-replay-execute; review cdc_poison_quarantine.json, cdc_quarantine_inspection.json, and cdc_replay_execution.json. | | CDC compare and repair | Source-to-sink current-state consistency, bounded repair planning, and offset-safe repair execution | uv run dpone ops cdc-compare-repair, then optionally uv run dpone ops cdc-repair-execute; review cdc_compare_repair.json, cdc_repair_plan.json, and cdc_repair_execution.json. | | CDC retention gap auto-resync | Source retention gap checks, bounded resync planning, and offset-safe resync execution | uv run dpone ops cdc-retention-check, then uv run dpone ops cdc-resync-plan and approved uv run dpone ops cdc-resync-execute; review cdc_retention_check.json, cdc_resync_plan.json, cdc_resync_actions.json, and cdc_resync_execution.json. | | CDC live runtime adapters | Opt-in vendor-live MSSQL -> ClickHouse CDC read, ClickHouse apply, and SQL-backed offset commit check | uv run dpone ops cdc-runtime-run --mode live, then review cdc_runtime_run.json. | | ClickHouse CDC materialization | Current-state serving table rebuild from ClickHouse append-only CDC logs | uv run dpone ops cdc-materialize-clickhouse, then review cdc_materialization.json. | | ClickHouse CDC typed materialization | Typed current-state serving table rebuild from ClickHouse append-only CDC logs, including schema drift and parse quarantine gates | uv run dpone ops cdc-materialize-clickhouse-typed --fail-on-parse-errors --schema-drift-mode strict, then review cdc_typed_materialization.json and cdc_typed_parse_quarantine.json. | | Release publish | Public release | Push annotated tag vX.Y.Z or manual release workflow. | | GitOps Airflow runner pack | Airflow, KubernetesExecutor, KubernetesPodOperator, KubernetesPodExecutor, custom dpone image, runtime-only Airflow Connection bridge, AIRFLOW_CONN_* Kubernetes Secret refs, ExternalSecret/env skeletons, cluster-doctor namespace/service-account/RBAC/Secret-key/ExternalSecret readiness, optional Airflow-in-runtime Docker target, sparse checkout, sparse git-sync initContainers, git-sync clone depth, git-sync partial clone filters, airflow-dags-dev artifact-loading helper UX, run-spec, runtime profile, pod contract, runtime pack, artifact index, preflight, final XCom outcome, outcome gate, runtime evidence, Kubernetes smoke, pod launch evidence, evidence bundle, or GitOps bundle handoff changes | uv run pytest tests/test_airflow_runtime_adapter.py tests/test_gitops_airflow_pack.py tests/test_gitops_airflow_artifact_index_preflight.py tests/test_gitops_airflow_connection_bridge_plan.py tests/test_gitops_airflow_cluster_doctor.py tests/test_gitops_airflow_git_sync_contract.py tests/test_gitops_airflow_runner_pack.py tests/test_gitops_airflow_k8s_smoke.py tests/test_gitops_airflow_pod_launch_evidence.py tests/test_gitops_airflow_evidence_bundle.py tests/test_cli_gitops_airflow_commands.py tests/test_gitops_airflow_docs_contract.py tests/test_gitops_schema_contracts.py -q, then dpone gitops airflow render, dpone gitops airflow runtime-profile --runner-policy release --git-sync-repo ... --git-sync-image registry.k8s.io/git-sync/git-sync:v4.7.0 --git-sync-filter blob:none --airflow-connection-bridge k8s_secret --airflow-runtime-mode runtime_only, dpone gitops airflow pod-contract, dpone gitops airflow connection-bridge-plan --artifact-dir .dpone/gitops/airflow, dpone gitops airflow artifact-index --artifact-dir .dpone/gitops/airflow, dpone gitops airflow pack --artifact-dir .dpone/gitops/airflow --mode verify --runner-policy release, dpone gitops airflow pod-doctor --artifact-dir .dpone/gitops/airflow --runner-policy release, dpone gitops airflow preflight --artifact-dir .dpone/gitops/airflow --runner-policy release, dpone gitops airflow cluster-doctor --artifact-dir .dpone/gitops/airflow --mode plan --runner-policy release, dpone gitops airflow k8s-smoke --mode plan --runner-policy release, dpone gitops airflow pod-watch --mode plan --runner-policy release, dpone gitops airflow outcome-gate .dpone/gitops/airflow/xcom-summary.json --required-status passed, dpone gitops airflow doctor --require-attestation --runner-policy release, dpone gitops airflow evidence-verify --require-all-steps, and dpone gitops airflow evidence-bundle --require-k8s-smoke --require-pod-launch-evidence against generated artifacts. Collect runtime-profile.json, pod-contract.json, pod-spec.yaml, kpo-kwargs.json, connection-bridge-plan.json, airflow-connections-secret.yaml, airflow-connections-externalsecret.yaml, airflow-connections.env.example, airflow-cluster-doctor.json, artifact-index.json, airflow-runtime-pack.json, xcom-summary.json, airflow_dag_factory.py, outcome_gate.py, run-spec.json, runtime-evidence.json, airflow-k8s-smoke.json, airflow-pod-launch-evidence.json, and airflow-evidence-bundle.json as release evidence. This default gate is credential-free and does not start Airflow or Kubernetes pods; run-spec-exec is covered with the static CommandRunner test runner and writes the final XCom outcome to --xcom-output. Run dpone gitops airflow cluster-doctor --mode live --require-external-secret-ready, dpone gitops airflow k8s-smoke --mode live, and dpone gitops airflow pod-watch --mode live only in opt-in Airflow/Kubernetes jobs with a credentialed cluster and the custom dpone image digest pinned. Git-sync and Airflow connection bridge artifacts must serialize only Secret names and key names; secret values stay in Kubernetes Secrets and must not appear in JSON, YAML, Markdown, or logs. |

For the deployable Kubernetes pack layer, extend the same GitOps Airflow gate with uv run pytest tests/test_gitops_airflow_k8s_manifests.py tests/test_gitops_airflow_admission_check.py -q, then run dpone gitops airflow k8s-manifests --artifact-dir .dpone/gitops/airflow and dpone gitops airflow admission-check --artifact-dir .dpone/gitops/airflow --mode plan --runner-policy release against generated artifacts. Collect airflow-k8s-manifests.json, airflow-k8s-manifests.yaml, and airflow-admission-check.json as release evidence. Use --gitops-controller argocd for Argo CD Application paths and --gitops-controller flux for Flux Kustomization paths so the generated YAML contains controller-aware ownership metadata while keeping Secret values out of artifacts. admission-check --mode live is opt-in only for credentialed Airflow/Kubernetes jobs and uses server-side kubectl apply --dry-run=server; ordinary OSS CI remains credential-free.

Secrets and credentials policy

  • Ordinary PR CI must not require vendor credentials.
  • integration_live and vendor certification jobs are opt-in only.
  • PyPI publishing should use Trusted Publishing through GitHub OIDC.
  • PYPI_API_TOKEN is a fallback secret only, not the preferred path.
  • If a token appears in chat, an issue, a PR, a commit, or a CI log, revoke and rotate it before any public release.
  • Secrets must never be written to artifacts; diagnostics must redact credentials.

Artifact policy

Artifact Produced by Purpose
coverage.xml CI Coverage evidence and trend input.
dist/ CI and Release Built sdist/wheel artifacts.
GitHub Pages artifact docs workflow Deployable documentation site.
docs/benchmarks/ OSS code quality benchmark workflow Public benchmark Markdown, customer trust-center snapshot, SVG visuals, and raw JSON evidence with freshness metadata, quality gates, history trends, Complexity & Boundary Discipline, Semantic Maintainability Deep Scan, Scoring Validity & Calibration, Scale Readiness & Growth Simulation, quality headroom, architecture runway, Language/repo normalization, Sensitivity analysis, Anti-gaming guardrails, Refactor ROI Roadmap, quality debt, Evidence Trust & Auditability, Public Evidence Integrity, Source Citation Verification, Benchmark v3 Release Readiness, Claims Ledger, Runtime Certification Matrix, Executable Certification, Golden Dataset Evidence, Run Ledger, Certification Gates, Quality Budget As Code, Evidence Warehouse Export, release readiness pack, claim-to-source matrix, Independent Analyzer Cross-Validation, External analyzer execution, public artifact redaction, claim coverage, stale analyzer values, stale source values, runtime-certification/latest/run-ledger.json, runtime-certification/latest/contract-checks.json, oss-code-quality-benchmark-history.json, oss-benchmark-provenance.json, oss-benchmark-release-readiness-2026-06-12.md, oss-benchmark-release-readiness-2026-06-12.json, oss-architecture-delta.svg, oss-complexity-boundary.svg, oss-semantic-maintainability.svg, oss-god-object-radar.svg, oss-score-calibration.svg, oss-score-sensitivity.svg, oss-normalized-vs-raw.svg, oss-scale-readiness.svg, oss-architecture-runway.svg, oss-quality-headroom.svg, oss-refactor-roi-roadmap.svg, oss-evidence-confidence.svg, oss-public-evidence-integrity.svg, oss-source-verification.svg, oss-release-readiness-seal.svg, oss-independent-validation.svg, oss-analyzer-confidence.svg, and manual runner context.
test_artifacts/oss-code-quality-benchmark/pr-comment.md OSS code quality benchmark workflow Ready-to-use PR summary with gate status, executable certification status, PR regression gate, score movement, freshness, architecture delta, Complexity & Boundary Discipline, Semantic Maintainability Deep Scan, SOLID/DI/Clean Code evidence, DRY/KISS responsibility signals, Scoring Validity & Calibration, Scale Readiness & Growth Simulation, quality headroom, architecture runway, Language/repo normalization, Sensitivity analysis, Anti-gaming guardrails, Refactor ROI Roadmap, quality debt, Evidence Trust & Auditability, Source Citation Verification, Independent Analyzer Cross-Validation, Analyzer command ledger, maintainability risk register, reproducibility manifest, optional LOC/SLOC cross-check, external analyzer status, and dpone architecture hotspots.
test_artifacts/integration_matrix/ Integration matrix workflow Matrix metadata, behavior artifacts, JUnit output.
test_artifacts/connectors/ Connector certification workflow Connector capability and certification evidence.
test_artifacts/observability/maturity/ Observability maturity workflow Prometheus, OpenTelemetry, metrics index, and SLO smoke evidence.
test_artifacts/production_maturity/ Production maturity workflow Aggregated GA readiness evidence, checksums, blockers, score, and Markdown release review.
test_artifacts/industrial_readiness/ Industrial readiness workflow Local matrix, correctness, reliability, performance, UX, governance, checksums, blockers, score, and Markdown release review.
test_artifacts/route_certification/ Manual or scheduled route certification jobs Normalized per-route evidence files plus embedded route readiness reports.
test_artifacts/route_certification_release/ Manual route certification release jobs Release-level route_certification_release.json, Markdown review, release notes fragment, route index, artifact index, blockers, warnings, and first-class route bundle checksums.
test_artifacts/route_release_finalize/ Manual route release finalizer jobs Final route_release_finalizer.json, Markdown runbook, nested route certification release report, history index, freshness/provenance/regression checks, blockers, and route scores.
test_artifacts/route_release/ Manual or scheduled route release jobs Route-scoped release go/no-go receipt with checksums, blockers, score, route identity, and release summary.
test_artifacts/route_readiness/ Manual or scheduled route certification jobs Per-route JSON/Markdown readiness reports with matrix-backed profiles and checksumed evidence.
test_artifacts/route_refresh/ Manual or scheduled route refresh planning jobs Per-route route_refresh_plan.json and Markdown Runbook for bounded backfill, replay, refresh, resync, approval, state rewind, and evidence checks.
test_artifacts/route_refresh_execution/ Manual or scheduled route refresh execution jobs Per-route route_refresh_execution.json and Markdown Runbook with dry-run/execute mode, chunk results, idempotency keys, blockers, and state-promotion readiness.
test_artifacts/route_refresh_snapshot_capture/ Manual or scheduled route refresh snapshot capture jobs Per-route route_refresh_snapshot_capture.json, source_route_refresh_snapshot.json, sink_route_refresh_snapshot.json, and Markdown Runbook with chunk capture status, typed hash inputs, blockers, and verification-ready paths.
test_artifacts/route_refresh_verification/ Manual or scheduled route refresh verification jobs Per-route route_refresh_verification.json and Markdown Runbook with source/sink row counts, boundaries, duplicate/null key counters, typed hash comparison, blockers, and state-promotion readiness.
test_artifacts/live_certification/refresh-executor/mssql-clickhouse/ Opt-in Docker-live MSSQL -> ClickHouse refresh executor certification Plan, executor config, first execution receipt, idempotency replay receipt, route_refresh_snapshot_capture.json, source/sink snapshot JSON, route_refresh_verification.json, per-chunk prepare/export/load artifacts, 10,000 rows, 200 columns, schema evolution, typed hash assertions, and JUnit evidence from test_mssql_clickhouse_refresh_executor_live_integration.py.
test_artifacts/live_certification/refresh-executor/postgres-mssql/ Opt-in Docker-live Postgres -> MSSQL refresh executor certification Plan, executor config, first execution receipt, idempotency replay receipt, route_refresh_snapshot_capture.json, source/sink snapshot JSON, route_refresh_verification.json, per-chunk prepare/export/load artifacts, 10,000 rows, 200 columns, schema evolution, typed hash assertions, and JUnit evidence from test_postgres_mssql_refresh_executor_live_integration.py.
test_artifacts/cdc_apply/ Manual or scheduled CDC apply certification jobs Per-fixture apply correctness, delete semantics, typed hash, generated handoff evidence, and embedded handoff reports.
test_artifacts/cdc_handoff/ Manual or scheduled CDC handoff jobs Per-stream CDC snapshot boundary, apply, retention, delete, typed hash, schema drift, and go/no-go reports.
test_artifacts/cdc_observability/ Manual or scheduled CDC observability evidence jobs Per-stream CDC lag, freshness, retention, offset commit, replay budget, throughput, blockers, and runbook artifacts.
test_artifacts/cdc_recovery/ Manual or scheduled CDC recovery evidence jobs Per-stream fault-injection scenario, restart/resume, offset ordering, idempotent replay, partial commit repair, poison quarantine, and retention recovery artifacts.
test_artifacts/cdc_schema/ Manual or scheduled CDC schema evolution evidence jobs Per-stream schema-change capture, compatibility, DDL dry-run, backfill, breaking-change approval, and offset/schema ordering artifacts.
test_artifacts/cdc_schema_apply/ Manual or scheduled CDC schema apply jobs Per-stream target DDL plan/apply reports, additive column backfill SQL, blockers, warnings, and optional typed refresh evidence.
test_artifacts/cdc_promotion/ Manual or scheduled CDC promotion jobs Final per-stream production_ready and promote_offsets decision artifacts plus normalized gate evidence.
test_artifacts/cdc_runtime/ Manual or scheduled CDC runtime smoke jobs Per-stream bounded read, sink apply receipt, durable checkpoint, blockers, and runtime metrics.
test_artifacts/cdc_poison_quarantine/ Manual or scheduled CDC poison replay jobs Per-stream poison quarantine, inspection, replay execution, duplicate replay metrics, blockers, and runbook artifacts.
test_artifacts/cdc_compare_repair/ Manual or scheduled CDC compare and repair jobs Per-stream source-to-sink compare reports, repair plans, repair execution receipts, blockers, warnings, and no-offset-mutation evidence.
test_artifacts/cdc_retention_resync/ Manual or scheduled CDC retention gap auto-resync jobs Per-stream retention checks, resync plans, replayable resync actions, execution receipts, blockers, warnings, and no-offset-mutation evidence.
test_artifacts/cdc_materialization/ Manual or scheduled ClickHouse CDC serving jobs Per-stream current-state materialization reports, delete-mode evidence, row counts, blockers, and serving table target metadata.
test_artifacts/cdc_typed_materialization/ Manual or scheduled ClickHouse CDC typed serving jobs Per-stream typed materialization reports, declared column inventory, delete-mode evidence, schema drift evidence, parse quarantine artifacts, row counts, blockers, and serving table target metadata.

What to do when CI/CD is red

Use Failure runbooks. The short version is:

  1. Identify the failing workflow and job, not just the failing commit status.
  2. Reproduce the exact command locally when the job is deterministic.
  3. Check whether the failure is default PR CI, manual matrix, scheduled certification, or release-only.
  4. Fix code, docs, tests, or workflow configuration in the same layer that failed.
  5. Add or update documentation when the failure teaches a new recovery path.
  6. Never bypass a red release, docs deploy, secret scan, or package metadata check for a public release.

Developer boundary

CI/CD changes are code changes. They must follow the same standards as runtime work: small diffs, least-privilege permissions, deterministic commands, explicit artifacts, and self-service docs. See Developer CI/CD guide.

Certification release summary

The final manual promotion gate is .github/workflows/certification-release-summary.yml. It downloads replay, source -> sink matrix, and connector certification artifacts from selected workflow run IDs, verifies their evidence chains, and publishes certification-release-summary as the go/no-go artifact for release review.

See the detailed workflow guide: CI/CD workflows.

Full certification automation

The recurring source -> sink release confidence gate is .github/workflows/full-certification.yml. It runs the matrix profile, benchmark baseline, run registry, OpenLineage export, evidence bundle, strategy bundle, certification suite, artifact index, and tamper-evident evidence chain in one scheduled/manual workflow.

See the detailed workflow guide: CI/CD workflows.

Observability maturity

The runtime telemetry confidence gate is .github/workflows/observability-maturity.yml. It runs the focused observability tests, exports prometheus_metrics.prom, opentelemetry_metrics.json, runtime_metrics.json, runtime_metrics.md, metrics_index.json, evaluates a local SLO smoke check, and uploads observability-maturity-report.

See the detailed workflow guide: CI/CD workflows.

Production maturity

The final release-readiness aggregator is .github/workflows/production-maturity.yml. It runs uv run dpone ops production-maturity, verifies required evidence domains, writes JSON and Markdown reports, indexes artifacts, and uploads production-maturity-report.

See the operator guide: Production maturity gate.

Industrial readiness

The next industrial maturity aggregator is .github/workflows/industrial-readiness.yml. It runs uv run dpone ops industrial-readiness, verifies local matrix, correctness, reliability, performance lab, UX, and governance evidence, writes JSON and Markdown reports, indexes artifacts, and uploads industrial-readiness-report.

See the operator guide: Industrial readiness gate.

OSS code quality benchmark

The public comparator benchmark is refreshed through .github/workflows/oss-code-quality-benchmark.yml. Run it manually with project=all for a complete refresh, or target one comparator when a single external checkout failed; accepted values are project=all, dpone, airbyte, dlt, pentaho-kettle, apache-hop, or sling. Use verify_source_urls=true only when you want live URL checks for Source Citation Verification; local files and URL metadata are always checked, and stale source values are retained beside the latest error when a live source refresh fails. The workflow installs the external analyzer execution toolchain (tokei, cloc, radon, lizard), runs executable certification with --run-certification --certification-mode local --fail-on-certification, passes the GitHub actor, run URL, branch, and SHA into the benchmark header, uses --previous-data to preserve stale values, uses --baseline-data for candidate-vs-baseline quality deltas, verifies public artifact redaction before upload, writes docs/benchmarks/data/oss-code-quality-benchmark-history.json, docs/benchmarks/data/oss-benchmark-provenance.json, docs/benchmarks/data/runtime-certification/latest/run-ledger.json, docs/benchmarks/data/runtime-certification/latest/contract-checks.json, and the Benchmark v3 Release Readiness pack, renders docs/benchmarks/assets/oss-architecture-delta.svg, docs/benchmarks/assets/oss-complexity-boundary.svg, docs/benchmarks/assets/oss-semantic-maintainability.svg, docs/benchmarks/assets/oss-god-object-radar.svg, docs/benchmarks/assets/oss-score-calibration.svg, docs/benchmarks/assets/oss-score-sensitivity.svg, docs/benchmarks/assets/oss-normalized-vs-raw.svg, docs/benchmarks/assets/oss-scale-readiness.svg, docs/benchmarks/assets/oss-architecture-runway.svg, docs/benchmarks/assets/oss-quality-headroom.svg, docs/benchmarks/assets/oss-refactor-roi-roadmap.svg, docs/benchmarks/assets/oss-evidence-confidence.svg, docs/benchmarks/assets/oss-public-evidence-integrity.svg, docs/benchmarks/assets/oss-source-verification.svg, docs/benchmarks/assets/oss-release-readiness-seal.svg, docs/benchmarks/assets/oss-independent-validation.svg, and docs/benchmarks/assets/oss-analyzer-confidence.svg, enforces dpone quality gates, uploads oss-code-quality-benchmark, writes a ready PR summary with executable scenario failures or stale scenario ages, a PR regression gate, Complexity & Boundary Discipline, Semantic Maintainability Deep Scan, SOLID/DI/Clean Code evidence, DRY/KISS responsibility signals, Scoring Validity & Calibration, Scale Readiness & Growth Simulation, quality headroom, architecture runway, Language/repo normalization, Sensitivity analysis, Anti-gaming guardrails, Refactor ROI Roadmap, quality debt, Evidence Trust & Auditability, Public Evidence Integrity, Source Citation Verification, Benchmark v3 Release Readiness, Claims Ledger, Runtime Certification Matrix, Executable Certification, Quality Budget As Code, Evidence Warehouse Export, claim-to-source matrix, claim coverage, Independent Analyzer Cross-Validation, Analyzer command ledger, reproducibility manifest, optional LOC/SLOC cross-check, external analyzer status, stale analyzer values, and a maintainability risk register, generates docs/benchmarks/dpone-trust-center-snapshot-2026-06-12.md plus its JSON and badge SVG, and can open a PR with the generated docs and raw JSON evidence.