Release evidence¶

Release evidence is the final go/no-go layer for public package publishing, GitHub release notes, and internal production promotion. It aggregates immutable artifacts from tests, certification, performance, state, reconciliation, docs, security, and supply-chain checks.

Contents¶

When it is required
Artifact contract
CLI workflow
Pre-release checklist
Runbook
Developer notes

When it is required¶

Certification profile	Release evidence role
`local_live`	Feature-level disposable-service confidence before broader release evidence is needed.
`real_local`	Required local release-candidate profile for minor and major releases. Produces `benchmark-slo-gate`, `performance-certification`, `live-state-reconciliation`, `release-evidence-pack`, and evidence-chain artifacts without vendor credentials.
`vendor_live`	Optional managed/provider proof when a release changes BigQuery, managed APIs, external Kafka, or other credentialed systems.
`connector_certification_schedule`	Async scheduled release-evidence gate for connector confidence. Record the workflow run ID, status, and artifacts in the release review when connector behavior is relevant.

Release type	Required gate
Patch with docs-only changes	Default CI plus docs strict gate.
Patch with runtime/CLI/connector changes	Default CI plus focused live or matrix evidence.
Minor release	Full release evidence pack with `real_local` certification.
Major release	Full release evidence pack with `real_local`; add `vendor_live` where external providers are affected.

For the current minor release line, do not tag or publish PyPI until release_evidence_pack.json is green.

Artifact contract¶

dpone ops release-evidence-pack defaults to these required artifacts:

Artifact name	Producer	Purpose
`service_markers`	local service marker tests	Confirms disposable Postgres, MSSQL, ClickHouse, Kafka, Schema Registry, and MinIO marker tests passed.
`certification_pack`	`dpone ops certification-pack`	Source -> sink matrix, observability, performance, and state/reconciliation evidence summary.
`performance_certification`	`dpone ops performance-certification`	Throughput, duration, memory, and failure-rate thresholds.
`live_state_reconciliation`	`dpone ops live-state-reconciliation`	State backends, XMin/Kafka/CDC state checks, and physical-delete reconciliation proof.
`pre_release_checklist`	`dpone ops pre-release-checklist`	CLI, run, Python API, lineage, matrix, contracts, docs, and package release checklist.
`evidence_chain`	`dpone ops evidence-chain` and `dpone ops evidence-chain-verify`	Tamper-evident artifact checksum chain.
`agent_governance_gate`	`tools/agent_policy/governance_gate.py`	Agent-control, release-attestation, Scorecard, and SLSA self-assessment receipt for governance or workflow changes.
`github_attestation_receipts`	GitHub CLI in `.github/workflows/release.yml`	Per-artifact GitHub Artifact Attestation verification receipts for wheel and source distributions.

Additional release gates can be attached with repeated --artifact name=path and made mandatory with repeated --require name.

Critical source -> sink release candidates should also attach the route-level live bundle:

dpone ops release-evidence-pack \
  --release vX.Y.Z-rc1 \
  --profile native_transfer \
  --artifact route_live_evidence_bundle=test_artifacts/route_live/mssql_to_clickhouse/route_live_certification.json \
  --require route_live_evidence_bundle \
  --output-dir test_artifacts/live_certification/release-evidence \
  --format json

When the route release candidate needs one reproducible release train, use dpone ops route-rc-orchestrator and attach both route_rc_orchestration.json and the nested release_evidence_pack.json. Run dpone ops route-rc-execute against that orchestration receipt to create route_rc_execution.json; dry-run receipts are acceptable for ordinary OSS CI, while --execute receipts are required when the release review claims a Docker-live command train was actually run.

CLI workflow¶

dpone ops live-certification-plan \
  --profile real_local \
  --row-count 25000 \
  --output-dir test_artifacts/live_certification/plan \
  --format json

The real_local gate starts disposable services through docker/docker-compose.integration.yml before producing the final evidence pack.

dpone ops performance-certification \
  --profile real_local \
  --row-count 25000 \
  --metrics-json '{"throughput_rows_per_second":1000,"duration_seconds":60,"memory_peak_mb":1024,"failure_rate":0}' \
  --minimum-json '{"throughput_rows_per_second":500,"failure_rate":0}' \
  --maximum-json '{"duration_seconds":120,"memory_peak_mb":2048}' \
  --output-dir test_artifacts/live_certification/performance-certification \
  --format json

dpone ops live-state-reconciliation \
  --profile real_local \
  --artifact state=test_artifacts/live_certification/state_evidence.json \
  --artifact reconciliation=test_artifacts/live_certification/reconciliation_evidence.json \
  --require state \
  --require reconciliation \
  --output-dir test_artifacts/live_certification/live-state-reconciliation \
  --format json

dpone ops release-evidence-pack \
  --release vX.Y.Z \
  --profile real_local \
  --artifact service_markers=test_artifacts/live_certification/service_markers.json \
  --artifact certification_pack=test_artifacts/live_certification/certification-pack/connector_certification_pack.json \
  --artifact performance_certification=test_artifacts/live_certification/performance-certification/performance_certification.json \
  --artifact live_state_reconciliation=test_artifacts/live_certification/live-state-reconciliation/live_state_reconciliation.json \
  --artifact pre_release_checklist=test_artifacts/live_certification/pre-release/pre_release_checklist.json \
  --artifact evidence_chain=test_artifacts/live_certification/evidence-chain/evidence_chain_index.json \
  --output-dir test_artifacts/live_certification/release-evidence \
  --format json

Pre-release checklist¶

Before a minor or major release, collect evidence for:

Area	Required proof
CLI UX	`dpone --help`, registered command smoke tests, JSON/Markdown output checks, and no unexpected non-zero exit for valid options.
Runtime execution	`dpone run --manifest ...` smoke and equivalent Python API execution through `dpone.api`.
Hierarchical lineage	Nested-object row IDs, parent row IDs, root row IDs, and list indexes are deterministic across retries.
Source -> sink strategies	Integration matrix artifacts for each supported source, sink, and strategy.
Docker-live routes	Opt-in MSSQL + ClickHouse and Postgres + MSSQL Docker evidence for release-candidate routes.
Contracts and guardrails	Schema contracts, runtime data contracts, policy checks, quarantine, and state-commit-after-load behavior.
Documentation	YAML examples parse, links are valid, MkDocs strict build passes, architecture/CI/CD/testing docs are current.
Package	Build, twine check, fresh install smoke, and PyPI version availability check.
Supply chain and agent governance	GitHub Artifact Attestation receipts, SLSA self-assessment, OSSF Scorecard signal, and `agent_governance_gate.json` when governance, workflow, release, or supply-chain controls changed.

Generate the machine-readable checklist:

dpone ops pre-release-checklist \
  --release vX.Y.Z \
  --release-type minor \
  --check cli_help_surface=true \
  --check cli_output_contracts=true \
  --check run_cli_manifest=true \
  --check run_python_api_manifest=true \
  --check nested_hierarchical_identity=true \
  --check nested_parent_child_integrity=true \
  --check source_sink_strategy_matrix=true \
  --check source_sink_artifacts=true \
  --check docker_live_routes=true \
  --check contracts_guardrails=true \
  --check documentation_yaml_examples=true \
  --check documentation_links=true \
  --check documentation_mkdocs=true \
  --check ci_cd_quality=true \
  --check package=true \
  --output-dir test_artifacts/live_certification/pre-release \
  --format json

Attach pre_release_checklist.json to release-evidence-pack for minor and major releases.

Runbook¶

Failure: performance_certification.not_passed.

Inspect the failed metric and threshold in performance_certification.json.
Check native fast paths, batch size, partitioning, staging backend, and target capacity.
Do not lower thresholds unless a reviewed benchmark-baseline update explains why.

Failure: live_state_reconciliation.not_passed.

Re-run state backend integration markers for Postgres and MSSQL.
Confirm XMin, Kafka offsets, CDC offsets, and run state advanced only after load success.
Re-run physical-delete reconciliation for changed/deleted rows.

Failure: release_evidence_pack.not_passed.

Open blockers in release_evidence_pack.json.
Missing required artifacts block release.
Red required artifacts must be fixed at the originating gate.
Do not edit generated evidence by hand.

Failure: pre_release_checklist.not_passed.

Open pre_release_checklist.json and fix every *.missing or *.not_passed blocker.
Re-run the actual gate, not only the checklist command.
Regenerate release-evidence-pack after the checklist is green.

Failure: docs are red.

Run uv run dpone docs check-docs.
Run uv run mkdocs build --strict.
Fix broken links, invalid YAML examples, outdated CLI reference, or stale architecture docs.

Developer notes¶

Concern	Module
Artifact validation	`dpone.ops.artifact_validation`
Performance evidence	`dpone.ops.performance_certification`
State/reconciliation evidence	`dpone.ops.live_state_reconciliation`
Release evidence pack	`dpone.ops.release_evidence_pack`
Pre-release checklist	`dpone.ops.pre_release_checklist`
Live certification plan	`dpone.ops.live_certification`

Keep release evidence services small and dependency-light. They summarize artifacts; they do not execute database loads directly.

Native transfer release profile¶

Use --profile native_transfer for release candidates that certify critical native transfer routes such as Postgres -> MSSQL or MSSQL -> ClickHouse. This profile keeps the standard release evidence requirements and additionally requires strategy_certification_bundle.

For MSSQL -> ClickHouse releases that change binary/native transfer behavior, also attach the BCP native wide-type certification evidence produced by tools/mssql_clickhouse_bcp_native_type_certification.py. The expected minimum release proof is 10,000 rows, 200 columns, all supported primitive/text/ binary/temporal families, exact typed hash equality, duplicate-key count 0, and eager cleanup under the configured worker storage policy.

dpone ops release-evidence-pack \
  --release vX.Y.Z \
  --profile native_transfer \
  --artifact service_markers=test_artifacts/release/service_markers.json \
  --artifact certification_pack=test_artifacts/live_certification/certification-pack/connector_certification_pack.json \
  --artifact performance_certification=test_artifacts/live_certification/performance-certification/performance_certification.json \
  --artifact live_state_reconciliation=test_artifacts/live_certification/live-state-reconciliation/live_state_reconciliation.json \
  --artifact pre_release_checklist=test_artifacts/pre-release/pre_release_checklist.json \
  --artifact evidence_chain=test_artifacts/evidence-chain/evidence_chain_index.json \
  --artifact strategy_certification_bundle=test_artifacts/strategy_certification/postgres_mssql/strategy_certification_bundle.json \
  --output-dir test_artifacts/release-evidence/native-transfer \
  --format json

If strategy_certification_bundle is missing, the pack is red with strategy_certification_bundle.missing. Use explicit repeated --require flags only for exceptional custom gates; otherwise let the profile choose the required artifact set.