CI/CD

dpone uses GitHub Actions as the public OSS automation path. The default branch for the public release flow is master.

This page is the CI/CD index. It explains the automation model at a high level and links to detailed workflow docs, failure runbooks, and developer guidance for changing CI/CD safely.

CI/CD map

flowchart LR
    Dev["Developer branch or fork"] --> PR["Pull request to master"]
    PR --> CI["CI quality matrix"]
    PR --> Docs["Docs strict build"]
    PR --> Security["Secret scan and CodeQL"]
    CI --> Review["Review and merge"]
    Docs --> Review
    Security --> Review
    Review --> Master["master"]
    Master --> Pages["GitHub Pages deploy"]
    Master --> Scorecard["OSSF Scorecard"]
    Master --> Manual["Manual integration matrix"]
    Manual --> Obs["Observability maturity"]
    Tag["vX.Y.Z tag"] --> Release["Build package and publish to PyPI"]
    Manual --> Cert["Connector certification artifacts"]
    Cert --> Maturity["Production maturity gate"]
    Obs --> Maturity
    Maturity --> Industrial["Industrial readiness gate"]
    Industrial --> Release
    Maturity --> Release

Detailed docs

Need	Doc
Understand each workflow, trigger, job, artifact, and local reproduction command	Workflow reference
Recover a red CI/CD run	Failure runbooks
Understand release publishing, GitHub Pages, and PyPI Trusted Publishing	Release and Pages automation
Run the manual source/sink matrix	Manual integration matrix and matrix runbook
Certify connectors	Connector certification
Add or change CI/CD as a developer	Developer CI/CD guide
Run the local/default test gates	Testing and Testing runbooks

Workflow inventory

Workflow	File	Trigger	Required by default	Purpose
CI	.github/workflows/ci.yml	push to master, pull request to master, manual dispatch	yes	Lint, format check, type check, non-live tests, coverage XML, package build, PostgreSQL XMin integration.
docs	.github/workflows/pages.yml	docs changes on push/PR, manual dispatch	yes for docs changes	Strict MkDocs build; deploys GitHub Pages from master.
Secret Scan	.github/workflows/secret-scan.yml	push, PR, weekly schedule	yes	TruffleHog verified secret scan.
CodeQL	.github/workflows/codeql.yml	push, PR, weekly schedule	yes	Python static security analysis.
OSSF Scorecard	.github/workflows/scorecard.yml	push to master, branch protection changes, weekly schedule	advisory/security	Supply-chain posture analysis and SARIF upload.
Release	.github/workflows/release.yml	vX.Y.Z tag, manual dispatch	release-only	Build sdist/wheel, run twine check, publish to PyPI through Trusted Publishing or token fallback.
Source/Sink integration matrix	.github/workflows/integration-matrix.yml	manual dispatch	manual	Runs all source -> sink strategy contracts or local/mock matrix layers with artifacts.
Observability maturity	.github/workflows/observability-maturity.yml	manual dispatch, weekly schedule	scheduled/manual	Exports Prometheus/OpenTelemetry metrics, validates SLO smoke, and publishes observability evidence.
Connector certification	.github/workflows/connector-certification.yml	daily schedule, manual dispatch	scheduled/manual	Offline certification, optional local-live connector markers, optional vendor-live tests.
Production maturity	.github/workflows/production-maturity.yml	weekly schedule, manual dispatch	scheduled/manual	Aggregates certification, CDC, performance, security, supply-chain, governance, and docs evidence into one release readiness report.
Industrial readiness	.github/workflows/industrial-readiness.yml	weekly schedule, manual dispatch	scheduled/manual	Aggregates local matrix, correctness, reliability, performance lab, UX, and governance evidence into one industrial readiness report.

Default PR gate

For normal code and documentation changes, contributors should be able to reproduce the default gate locally:

uv sync --all-extras
uv run ruff check .
uv run ruff format --check .
uv run mypy --config-file mypy.ini
uv run pytest -m "not integration_live" --cov=src/dpone --cov-report=xml
uv build

For docs changes, also run:

python -m pip install -r docs/requirements.txt
mkdocs build --strict

For release candidates, add:

uv tool run twine check dist/*

GitHub Actions runtime baseline

All first-party and core third-party workflow actions should stay on Node 24-compatible major versions. This prevents GitHub Actions deprecation warnings from leaking into release, Pages, and certification gates.

Current baseline:

Action family	Minimum major used by dpone	Reason
actions/checkout	v6	Node 24 action runtime.
actions/setup-python	v6	Node 24 action runtime.
astral-sh/setup-uv	v7	Node 24 action runtime.
actions/upload-artifact	v6	Node 24 action runtime.
actions/upload-pages-artifact	v5	Pages artifact upload on the current runtime line.
actions/configure-pages	v6	Pages setup on the current runtime line.
actions/deploy-pages	v5	Pages deploy on the current runtime line.
github/codeql-action/*	v4	Supported CodeQL line with current runtime support.
softprops/action-gh-release	v3	Node 24 action runtime for GitHub Releases.

Self-hosted runners must be new enough to support Node 24 actions. If a workflow fails immediately during action startup, update the runner first, then re-run the workflow before changing dpone code.

Manual and scheduled gates

Manual/scheduled gates are intentionally separate from ordinary PR CI because they may start services, use external credentials, or run longer certification suites.

Gate	When to run	Command or workflow
Source/sink matrix mock_contract	Docs/manifest/strategy support changes	GitHub Actions Source/Sink integration matrix with run_mode=mock_contract.
Source/sink matrix mock_local	Connector or staging behavior changes	GitHub Actions Source/Sink integration matrix with run_mode=mock_local.
Real-local release certification	Minor/major release candidates	GitHub Actions Live certification with profile=real_local; requires green release-evidence-pack.
Vendor live certification	Release candidates and managed connector changes	Connector certification with run_vendor_live=true.
Connector local-live certification	Connector runtime changes	Scheduled/manual Connector certification.
Observability maturity	Metrics, SLO, run-report, or artifact evidence changes	GitHub Actions Observability maturity.
Production maturity	Release candidates and weekly readiness review	GitHub Actions Production maturity or uv run dpone ops production-maturity.
Industrial readiness	Industrial release candidates and readiness review	GitHub Actions Industrial readiness or uv run dpone ops industrial-readiness.
Release publish	Public release	Push annotated tag vX.Y.Z or manual release workflow.

Secrets and credentials policy

• Ordinary PR CI must not require vendor credentials.
• integration_live and vendor certification jobs are opt-in only.
• PyPI publishing should use Trusted Publishing through GitHub OIDC.
• PYPI_API_TOKEN is a fallback secret only, not the preferred path.
• If a token appears in chat, an issue, a PR, a commit, or a CI log, revoke and rotate it before any public release.
• Secrets must never be written to artifacts; diagnostics must redact credentials.

Artifact policy

Artifact	Produced by	Purpose
coverage.xml	CI	Coverage evidence and trend input.
dist/	CI and Release	Built sdist/wheel artifacts.
GitHub Pages artifact	docs workflow	Deployable documentation site.
test_artifacts/integration_matrix/	Integration matrix workflow	Matrix metadata, behavior artifacts, JUnit output.
test_artifacts/connectors/	Connector certification workflow	Connector capability and certification evidence.
test_artifacts/observability/maturity/	Observability maturity workflow	Prometheus, OpenTelemetry, metrics index, and SLO smoke evidence.
test_artifacts/production_maturity/	Production maturity workflow	Aggregated GA readiness evidence, checksums, blockers, score, and Markdown release review.
test_artifacts/industrial_readiness/	Industrial readiness workflow	Local matrix, correctness, reliability, performance, UX, governance, checksums, blockers, score, and Markdown release review.

What to do when CI/CD is red

Use Failure runbooks. The short version is:

1. Identify the failing workflow and job, not just the failing commit status.
2. Reproduce the exact command locally when the job is deterministic.
3. Check whether the failure is default PR CI, manual matrix, scheduled certification, or release-only.
4. Fix code, docs, tests, or workflow configuration in the same layer that failed.
5. Add or update documentation when the failure teaches a new recovery path.
6. Never bypass a red release, docs deploy, secret scan, or package metadata check for a public release.

Developer boundary

CI/CD changes are code changes. They must follow the same standards as runtime work: small diffs, least-privilege permissions, deterministic commands, explicit artifacts, and self-service docs. See Developer CI/CD guide.

Certification release summary

The final manual promotion gate is .github/workflows/certification-release-summary.yml. It downloads replay, source -> sink matrix, and connector certification artifacts from selected workflow run IDs, verifies their evidence chains, and publishes certification-release-summary as the go/no-go artifact for release review.

See the detailed workflow guide: CI/CD workflows.

Full certification automation

The recurring source -> sink release confidence gate is .github/workflows/full-certification.yml. It runs the matrix profile, benchmark baseline, run registry, OpenLineage export, evidence bundle, strategy bundle, certification suite, artifact index, and tamper-evident evidence chain in one scheduled/manual workflow.

See the detailed workflow guide: CI/CD workflows.

Observability maturity

The runtime telemetry confidence gate is .github/workflows/observability-maturity.yml. It runs the focused observability tests, exports prometheus_metrics.prom, opentelemetry_metrics.json, runtime_metrics.json, runtime_metrics.md, metrics_index.json, evaluates a local SLO smoke check, and uploads observability-maturity-report.

See the detailed workflow guide: CI/CD workflows.

Production maturity

The final release-readiness aggregator is .github/workflows/production-maturity.yml. It runs uv run dpone ops production-maturity, verifies required evidence domains, writes JSON and Markdown reports, indexes artifacts, and uploads production-maturity-report.

See the operator guide: Production maturity gate.

Industrial readiness

The next industrial maturity aggregator is .github/workflows/industrial-readiness.yml. It runs uv run dpone ops industrial-readiness, verifies local matrix, correctness, reliability, performance lab, UX, and governance evidence, writes JSON and Markdown reports, indexes artifacts, and uploads industrial-readiness-report.

See the operator guide: Industrial readiness gate.