Developer CI/CD guide¶

This page explains how to add or change CI/CD safely in dpone. It belongs in Developer docs because workflow changes are implementation changes: they affect release safety, contributor UX, security posture, and public artifacts.

Principles¶

Keep the default PR gate deterministic and credential-free.
Keep vendor credentials in manual or scheduled gates only.
Use least-privilege workflow permissions.
Prefer explicit artifacts over hidden logs.
Make every new workflow reproducible locally or document why it cannot be.
Update user docs and runbooks in the same PR as workflow changes.
Keep master as the default branch in workflow triggers and docs.

When to add a new workflow¶

Add a workflow only when one of these is true:

Case	Prefer
Fast deterministic quality check	Extend `.github/workflows/ci.yml`.
Documentation build/deploy behavior	Extend `.github/workflows/pages.yml`.
Public package publishing	Extend `.github/workflows/release.yml`.
Long-running source/sink or connector certification	Manual/scheduled workflow.
Vendor credentials required	Manual/scheduled workflow with explicit secrets and docs.
Security/supply-chain posture	Dedicated security workflow with least-privilege permissions.

Do not add a new always-on PR workflow for slow, flaky, vendor-dependent, or credential-dependent tests.

New workflow checklist¶

Before opening a PR:

Choose the trigger: push, pull_request, workflow_dispatch, schedule, or tag.
Choose minimum permissions. Start with contents: read.
Decide whether the workflow must run on master, PRs, tags, schedule, or manual dispatch.
Add local reproduction commands to Workflow reference.
Add failure recovery to Failure runbooks.
Add artifact names and retention expectations.
Add or update tests that protect the docs/workflow contract.
Confirm no job writes secrets to logs or artifacts.

Changing the default CI gate¶

Default CI changes affect all contributors. Keep them boring and reproducible.

Required local gate:

uv sync --all-extras
uv run ruff check .
uv run ruff format --check .
uv run mypy --config-file mypy.ini
uv run pytest -m "not integration_live" --cov=src/dpone --cov-report=xml
uv build

Rules:

Do not require Docker for the default quality matrix unless the job is isolated like postgres-xmin.
Do not require cloud/vendor credentials.
Keep optional integration markers opt-in.
If coverage minimum changes, document the rationale in the PR.

Adding a manual integration gate¶

Use this pattern for broad matrix, stress, benchmark, or connector certification workflows:

on:
  workflow_dispatch:
    inputs:
      run_mode:
        type: choice
        options: [mock_contract, mock_local, vendor_live]

permissions:
  contents: read

Design requirements:

All input names must be documented.
Every run writes artifacts even on failure with if: always().
mock_contract and mock_local must not require external vendor credentials.
vendor_live must clearly list required secrets.
Use focused filters such as DPONE_MATRIX_CASE_ID for debugging.

Adding secrets¶

Before adding a secret:

Ask whether Trusted Publishing, OIDC, or local mock mode can avoid the secret.
Scope the secret to the smallest provider/project/package possible.
Use it only in the workflow/job that needs it.
Redact diagnostics and artifacts.
Add rotation instructions to the relevant runbook.

Never add secrets to ordinary PR CI from forks.

Adding artifacts¶

Artifacts should answer: what ran, with what inputs, what passed/failed, and what evidence exists.

Adding certification suite automation¶

Use Certification suite automation when a workflow needs to combine matrix correctness, benchmark regression, lineage, dbt lineage, and evidence checksums.

Recommended flow:

uv run dpone ops certification-run \
  --artifact-dir test_artifacts/certification/current

uv run dpone ops benchmark-baseline \
  --output-dir test_artifacts/benchmarks/current \
  --metrics-json "$METRICS_JSON" \
  --baseline-json "$BASELINE_JSON"

uv run dpone ops certification-suite \
  --output-dir test_artifacts/certification/suite \
  --suite-id "$GITHUB_RUN_ID" \
  --certification-report test_artifacts/certification/current/certification_report.json \
  --benchmark-baseline test_artifacts/benchmarks/current/benchmark_baseline.json \
  --require-benchmark \
  --format json

Rules:

Keep this workflow manual or scheduled unless it is credential-free and fast.
Upload test_artifacts/certification/suite/ with if: always().
For critical source -> sink routes, generate route evidence first and then run uv run dpone ops route-certification-pack; upload test_artifacts/route_certification/ and any embedded test_artifacts/route_readiness/ outputs with if: always().
For new or changed source -> sink onboarding UX, run uv run dpone ops connection-doctor, uv run dpone ops source-discover, uv run dpone ops route-bootstrap, and uv run dpone ops route-doctor against credential-free schema fixtures. Upload test_artifacts/onboarding/ with if: always() and treat route_doctor.json as the pre-readiness user-facing go/no-go artifact; see Route bootstrap and doctor.
For route acceptance changes, run uv run dpone ops route-conformance run for each changed source -> sink -> strategy pair and then uv run dpone ops route-conformance release-gate over the produced artifacts. Upload test_artifacts/route_conformance/ with if: always(). Ordinary OSS CI can use deterministic credential-free synthetic datasets; Docker-live/vendor-live jobs should attach live evidence separately. See Route Conformance Lab.
For protected Docker-live route acceptance, run DPONE_VENDOR_LIVE=1 DPONE_LIVE_ROUTE_CONFORMANCE=1 uv run dpone ops route-conformance live-run with --adapter vendor_live or --adapter docker against disposable Postgres, MSSQL, and ClickHouse services. The built-in protected bindings cover postgres -> mssql and mssql -> clickhouse for incremental_merge. Upload route_conformance_live.json, live_source_snapshot.json, live_sink_snapshot.json, and the embedded route_conformance.json with if: always(). Keep the default adapter in_memory for ordinary OSS CI and register real Postgres, MSSQL, or ClickHouse adapters only in opt-in jobs.
For route release candidates, run uv run dpone ops route-rc-execute without --execute first and upload route_rc_execution.json. Add --execute only behind an explicit manual workflow input and after Docker or vendor-live services are ready.
When retry/resume, state commit, fencing, repair, resync, or route execution ordering changes, run uv run dpone ops route-execution-ledger for the affected route and upload test_artifacts/route_execution/ with if: always(). The required route readiness evidence is route_execution_ledger. Use --store-backend sqlite --store-uri test_artifacts/route_execution/shared/route_execution_ledger.sqlite3 when a workflow has more than one local runner or needs atomic compare-and-swap lease fencing evidence; see Route execution ledger.
When source state, offset, xmin, LSN, cursor, or checkpoint promotion changes, run uv run dpone ops route-state-promote after route execution ledger evidence and upload test_artifacts/route_state/ with if: always(). The required route readiness evidence is state_promotion. Use --state-backend sqlite --state-uri test_artifacts/route_state/shared/route_state_store.sqlite3 when a workflow has more than one local runner or needs atomic compare-and-swap state evidence; see Route state promotion.
For route release candidates, run uv run dpone ops route-release-gate after route_readiness.json, route_certification_pack.json, route_execution_ledger.json, state_promotion.json, and any route-specific benchmark/schema/CDC evidence exist. Upload test_artifacts/route_release/ with if: always() and treat route_release_gate.json as the final route-scoped go/no-go artifact; see Route release gate.
For Docker-live or vendor-live route release candidates, run uv run dpone ops route-live-certification after the upstream live checks have produced route evidence. Upload test_artifacts/route_live/ with if: always() and pass the live bundle to route-release-gate:

--artifact route_live_evidence_bundle=<route_live_certification.json> \
--require route_live_evidence_bundle

See Route live certification. - For route releases, run uv run dpone ops route-certify after refresh execution, snapshot capture, exact verification, readiness, checklist, and evidence-chain artifacts exist. Use --profile oss_ci in ordinary CI and --profile vendor_live only in opt-in Docker-live/vendor-live jobs that also attach route_live_evidence_bundle. Upload test_artifacts/route_certify/ with if: always() and treat route_certification_bundle.json as the final route promotion artifact; see Route certify. - For release candidates that claim first-class route certification, run uv run dpone ops route-certify-release after every required route has a route_certification_bundle.json. Upload test_artifacts/route_certification_release/ with if: always() and treat route_certification_release.json as the release-level go/no-go artifact; see Route certify release. - For final route-certified release review, run uv run dpone ops route-release-finalize after route bundles are in a stable bundle root. Upload test_artifacts/route_release_finalize/ with if: always() and treat route_release_finalizer.json plus route_certification_history_index.json as the final route release evidence; see Route release finalize. - For route release candidates that need the full train in one receipt, run uv run dpone ops route-rc-orchestrator. Upload test_artifacts/route_rc/ with if: always() and treat route_rc_orchestration.json plus the nested release_evidence_pack.json as the review bundle; see Route release candidate orchestrator. - For release candidates that merge a stacked train, export every PR with gh pr view --json number,title,baseRefName,headRefName,state,mergeStateStatus,isDraft,url,statusCheckRollup, run uv run dpone ops release-rc-collect, and then run the generated uv run dpone ops release-rc-finalize command after route, docs, package, and release evidence exists. Upload test_artifacts/release_rc_collect/ and test_artifacts/release_rc_finalizer/ with if: always() and block tags unless release_rc_finalizer.json is rc_ready; see Release RC collector and Release RC finalizer. - For CDC promotion routes, generate or export a credential-free fixture, run uv run dpone ops cdc-apply-certification, then run or review the embedded uv run dpone ops cdc-handoff report; upload test_artifacts/cdc_apply/ with if: always(). - When CDC evidence is produced by another specialized job, generate snapshot boundary, CDC window, retention, apply correctness, delete semantics, typed hash, and schema drift artifacts first, then run uv run dpone ops cdc-handoff; upload test_artifacts/cdc_handoff/ with if: always(). - When CDC telemetry or SLO behavior changes, run uv run dpone ops cdc-observability-evidence with the handoff JSON, apply certification JSON, metrics JSON, and optional SLO profile; upload test_artifacts/cdc_observability/ with if: always(). - When CDC recovery behavior changes, run uv run dpone ops cdc-recovery-evidence with handoff, apply, observability, scenario, and optional policy JSON; upload test_artifacts/cdc_recovery/ with if: always(). - When CDC schema evolution or target DDL governance changes, run uv run dpone ops cdc-schema-evolution-evidence with handoff, apply, observability, recovery, schema-change, and optional policy JSON; upload test_artifacts/cdc_schema/ with if: always(). - When CDC target DDL apply or typed serving refresh behavior changes, run uv run dpone ops cdc-schema-apply --mode dry_run and, for approved local Docker routes, uv run dpone ops cdc-schema-apply --mode apply; upload test_artifacts/cdc_schema_apply/ with if: always(); see CDC schema apply. The opt-in live check must verify additive column DDL, backfill SQL, cdc_schema_apply_result.json, and typed_refresh/cdc_typed_materialization.json when --typed-refresh is set. - When a CDC stream is ready for release or environment promotion, run uv run dpone ops cdc-promotion-gate with apply, handoff, observability, recovery, and schema evolution JSON; upload test_artifacts/cdc_promotion/ with if: always(). The gate records production_ready and promote_offsets; it does not promote offsets itself. - When runtime CDC apply behavior changes, run uv run dpone ops cdc-runtime-run with a bounded events JSON and local checkpoint JSON; upload test_artifacts/cdc_runtime/ with if: always(). This verifies that offsets commit only after durable sink apply. - When CDC poison classification, quarantine, or replay behavior changes, run uv run dpone ops cdc-runtime-run --poison-mode quarantine_and_continue, uv run dpone ops cdc-quarantine-inspect, and uv run dpone ops cdc-replay-execute; upload test_artifacts/cdc_poison_quarantine/ with if: always(). See CDC poison quarantine and replay. The check must prove cdc_poison_quarantine.json, cdc_quarantine_inspection.json, cdc_replay_execution.json, duplicate_events_skipped, and no offset mutation during replay. - When CDC compare, repair planning, or repair execution changes, run uv run dpone ops cdc-compare-repair and, for approved bounded plans, uv run dpone ops cdc-repair-execute; upload test_artifacts/cdc_compare_repair/ with if: always(). See CDC compare and repair. The check must prove cdc_compare_repair.json, cdc_repair_plan.json, cdc_repair_execution.json, source-to-ClickHouse CDC log current-state comparison, and no offset mutation during repair. - When CDC retention gap auto-resync changes, run uv run dpone ops cdc-retention-check, uv run dpone ops cdc-resync-plan, and, for approved bounded plans, uv run dpone ops cdc-resync-execute; upload test_artifacts/cdc_retention_resync/ with if: always(). See CDC retention gap auto-resync. The check must prove cdc_retention_check.json, cdc_resync_plan.json, cdc_resync_actions.json, cdc_resync_execution.json, Docker-live mssql -> clickhouse probe coverage when credentials are available, and no offset mutation during resync. - When live CDC runtime adapters change, run the local smoke above and an opt-in vendor-live uv run dpone ops cdc-runtime-run --mode live check for mssql -> clickhouse; see CDC live runtime adapters. The live check must verify SQL Server read, ClickHouse apply, and no offset commit on non-durable sink receipts. - When ClickHouse CDC serving materialization changes, run uv run dpone ops cdc-materialize-clickhouse against a fixture or opt-in Docker route and upload test_artifacts/cdc_materialization/ with if: always(); see ClickHouse CDC materialization. The check must verify active-row and tombstone delete modes. - When ClickHouse CDC typed serving materialization changes, run uv run dpone ops cdc-materialize-clickhouse-typed against a fixture or opt-in Docker route and upload test_artifacts/cdc_typed_materialization/ with if: always(); see ClickHouse CDC typed materialization. The check must verify declared column types, active-row mode, tombstone mode, --fail-on-parse-errors, --schema-drift-mode strict, and cdc_typed_parse_quarantine.json for malformed payloads. - Upload test_artifacts/observability/ with if: always() when a workflow exports runtime metrics. - Add required secrets only to vendor-live jobs. - Update Developer certification suite when adding evidence types.

Adding observability artifacts¶

Use Runtime observability when a workflow needs Prometheus/OpenTelemetry evidence for a run.

Recommended flow:

uv run dpone run "$MANIFEST" --format json > test_artifacts/runs/run_report.json

uv run dpone observability metrics-export \
  --run-report test_artifacts/runs/run_report.json \
  --output-dir test_artifacts/observability/current \
  --label ci_run_id="$GITHUB_RUN_ID" \
  --label branch="$GITHUB_REF_NAME" \
  --format json

Rules:

Keep labels secret-free and low-cardinality.
Keep collector upload optional; artifact generation must work locally.
Link any new failure mode to Failure runbooks.
Update Developer observability guide when adding renderers or metric sources.

Adding supply-chain evidence¶

Use Supply-chain evidence when a release workflow needs SBOM, provenance, signature envelope, and checksum evidence.

Recommended flow:

uv build
uv run twine check dist/*

subjects=()
for artifact in dist/*.whl dist/*.tar.gz; do
  subjects+=(--subject "$artifact")
done

uv run dpone supply-chain attest \
  --release "$GITHUB_REF_NAME" \
  "${subjects[@]}" \
  --repository "$GITHUB_SERVER_URL/$GITHUB_REPOSITORY" \
  --commit-sha "$GITHUB_SHA" \
  --builder-id "github-actions:$GITHUB_RUN_ID" \
  --signing-key "$DPONE_LOCAL_ATTESTATION_KEY" \
  --signing-key-id github-actions \
  --output-dir test_artifacts/supply-chain/current \
  --format json

mkdir -p test_artifacts/supply-chain/github-attestations
for artifact in dist/*.whl dist/*.tar.gz; do
  gh attestation verify "$artifact" \
    --repo "$GITHUB_REPOSITORY" \
    --source-ref "$GITHUB_REF" \
    --source-digest "$GITHUB_SHA" \
    --format json \
    > "test_artifacts/supply-chain/github-attestations/$(basename "$artifact").github-attestation.json"
done

Rules:

Upload test_artifacts/supply-chain/ with if: always().
Keep DPONE_LOCAL_ATTESTATION_KEY in the release environment secrets and fail the release when it is missing.
Do not store local HMAC signing keys in artifacts or logs.
Use and verify GitHub Artifact Attestations or Sigstore/cosign for public identity-backed trust before PyPI publication.
Run uv run python tools/agent_policy/governance_gate.py --base-ref origin/master --output test_artifacts/agent-policy/agent_governance_gate.json when CI/CD, agent policy, release evidence, or supply-chain controls change.
Update Developer supply-chain guide when adding formats or signers.

Docs contract tests¶

When CI/CD documentation changes, add or update docs contract tests. A good test verifies:

public workflow files are named in docs;
default branch remains master;
runbooks mention every required workflow;
developer docs link to CI/CD implementation guidance.

PR description template for CI/CD changes¶

## CI/CD change

- Workflow(s):
- Trigger(s):
- Permissions:
- Secrets required:
- Artifacts produced:
- Local reproduction:
- Failure runbook updated:
- Docs updated:

Developer CI/CD guide¶

Principles¶

When to add a new workflow¶

New workflow checklist¶

Changing the default CI gate¶

Adding a manual integration gate¶

Adding secrets¶

Adding artifacts¶

Adding certification suite automation¶

Adding observability artifacts¶

Adding supply-chain evidence¶

Docs contract tests¶

PR description template for CI/CD changes¶

Related docs¶