Release evidence¶

Release evidence is the final go/no-go layer for public package publishing, GitHub release notes, and internal production promotion. It aggregates immutable artifacts from tests, certification, performance, state, reconciliation, docs, security, and supply-chain checks.

Contents¶

When it is required
Artifact contract
CLI workflow
Pre-release checklist
Runbook
Developer notes

When it is required¶

Certification profile	Release evidence role
`local_live`	Feature-level disposable-service confidence before broader release evidence is needed.
`real_local`	Required local release-candidate profile for minor and major releases. Produces `benchmark-slo-gate`, `performance-certification`, `live-state-reconciliation`, `release-evidence-pack`, and evidence-chain artifacts without vendor credentials.
`vendor_live`	Optional managed/provider proof when a release changes BigQuery, managed APIs, external Kafka, or other credentialed systems.

Release type	Required gate
Patch with docs-only changes	Default CI plus docs strict gate.
Patch with runtime/CLI/connector changes	Default CI plus focused live or matrix evidence.
Minor release	Full release evidence pack with `real_local` certification.
Major release	Full release evidence pack with `real_local`; add `vendor_live` where external providers are affected.

For the current minor release line, do not tag or publish PyPI until release_evidence_pack.json is green.

Artifact contract¶

dpone ops release-evidence-pack defaults to these required artifacts:

Artifact name	Producer	Purpose
`service_markers`	local service marker tests	Confirms disposable Postgres, MSSQL, ClickHouse, Kafka, Schema Registry, and MinIO marker tests passed.
`certification_pack`	`dpone ops certification-pack`	Source -> sink matrix, observability, performance, and state/reconciliation evidence summary.
`performance_certification`	`dpone ops performance-certification`	Throughput, duration, memory, and failure-rate thresholds.
`live_state_reconciliation`	`dpone ops live-state-reconciliation`	State backends, XMin/Kafka/CDC state checks, and physical-delete reconciliation proof.
`pre_release_checklist`	`dpone ops pre-release-checklist`	CLI, run, Python API, lineage, matrix, contracts, docs, and package release checklist.
`evidence_chain`	`dpone ops evidence-chain` and `dpone ops evidence-chain-verify`	Tamper-evident artifact checksum chain.

Additional release gates can be attached with repeated --artifact name=path and made mandatory with repeated --require name.

CLI workflow¶

dpone ops live-certification-plan \
  --profile real_local \
  --row-count 25000 \
  --output-dir test_artifacts/live_certification/plan \
  --format json

The real_local gate starts disposable services through docker/docker-compose.integration.yml before producing the final evidence pack.

dpone ops performance-certification \
  --profile real_local \
  --row-count 25000 \
  --metrics-json '{"throughput_rows_per_second":1000,"duration_seconds":60,"memory_peak_mb":1024,"failure_rate":0}' \
  --minimum-json '{"throughput_rows_per_second":500,"failure_rate":0}' \
  --maximum-json '{"duration_seconds":120,"memory_peak_mb":2048}' \
  --output-dir test_artifacts/live_certification/performance-certification \
  --format json

dpone ops live-state-reconciliation \
  --profile real_local \
  --artifact state=test_artifacts/live_certification/state_evidence.json \
  --artifact reconciliation=test_artifacts/live_certification/reconciliation_evidence.json \
  --require state \
  --require reconciliation \
  --output-dir test_artifacts/live_certification/live-state-reconciliation \
  --format json

dpone ops release-evidence-pack \
  --release v0.7.1 \
  --profile real_local \
  --artifact service_markers=test_artifacts/live_certification/service_markers.json \
  --artifact certification_pack=test_artifacts/live_certification/certification-pack/connector_certification_pack.json \
  --artifact performance_certification=test_artifacts/live_certification/performance-certification/performance_certification.json \
  --artifact live_state_reconciliation=test_artifacts/live_certification/live-state-reconciliation/live_state_reconciliation.json \
  --artifact pre_release_checklist=test_artifacts/live_certification/pre-release/pre_release_checklist.json \
  --artifact evidence_chain=test_artifacts/live_certification/evidence-chain/evidence_chain_index.json \
  --output-dir test_artifacts/live_certification/release-evidence \
  --format json

Pre-release checklist¶

Before a minor or major release, collect evidence for:

Area	Required proof
CLI UX	`dpone --help`, registered command smoke tests, JSON/Markdown output checks, and no unexpected non-zero exit for valid options.
Runtime execution	`dpone run --manifest ...` smoke and equivalent Python API execution through `dpone.api`.
Hierarchical lineage	Nested-object row IDs, parent row IDs, root row IDs, and list indexes are deterministic across retries.
Source -> sink strategies	Integration matrix artifacts for each supported source, sink, and strategy.
Contracts and guardrails	Schema contracts, runtime data contracts, policy checks, quarantine, and state-commit-after-load behavior.
Documentation	YAML examples parse, links are valid, MkDocs strict build passes, architecture/CI/CD/testing docs are current.
Package	Build, twine check, fresh install smoke, and PyPI version availability check.

Generate the machine-readable checklist:

dpone ops pre-release-checklist \
  --release v0.7.1 \
  --release-type minor \
  --check cli_commands=true \
  --check run_cli=true \
  --check run_python_api=true \
  --check hierarchical_lineage=true \
  --check source_sink_matrix=true \
  --check contracts_guardrails=true \
  --check documentation=true \
  --check package=true \
  --output-dir test_artifacts/live_certification/pre-release \
  --format json

Attach pre_release_checklist.json to release-evidence-pack for minor and major releases.

Runbook¶

Failure: performance_certification.not_passed.

Inspect the failed metric and threshold in performance_certification.json.
Check native fast paths, batch size, partitioning, staging backend, and target capacity.
Do not lower thresholds unless a reviewed benchmark-baseline update explains why.

Failure: live_state_reconciliation.not_passed.

Re-run state backend integration markers for Postgres and MSSQL.
Confirm XMin, Kafka offsets, CDC offsets, and run state advanced only after load success.
Re-run physical-delete reconciliation for changed/deleted rows.

Failure: release_evidence_pack.not_passed.

Open blockers in release_evidence_pack.json.
Missing required artifacts block release.
Red required artifacts must be fixed at the originating gate.
Do not edit generated evidence by hand.

Failure: pre_release_checklist.not_passed.

Open pre_release_checklist.json and fix every *.missing or *.not_passed blocker.
Re-run the actual gate, not only the checklist command.
Regenerate release-evidence-pack after the checklist is green.

Failure: docs are red.

Run uv run dpone docs check-docs.
Run uv run mkdocs build --strict.
Fix broken links, invalid YAML examples, outdated CLI reference, or stale architecture docs.

Developer notes¶

Concern	Module
Artifact validation	`dpone.ops.artifact_validation`
Performance evidence	`dpone.ops.performance_certification`
State/reconciliation evidence	`dpone.ops.live_state_reconciliation`
Release evidence pack	`dpone.ops.release_evidence_pack`
Pre-release checklist	`dpone.ops.pre_release_checklist`
Live certification plan	`dpone.ops.live_certification`

Keep release evidence services small and dependency-light. They summarize artifacts; they do not execute database loads directly.