Skip to content

Release

This document describes the public OSS release flow for dpone.

Prerequisites

  • GitHub repository: PaulKov/dpone.
  • PyPI project: dpone.
  • License: Apache-2.0.
  • Preferred publishing mode: PyPI Trusted Publishing through GitHub Actions.
  • Fallback publishing mode: short-lived PyPI API token stored as PYPI_API_TOKEN. Tag releases use this token automatically when it is configured. This is required for the first dpone-native-accel release unless a PyPI pending trusted publisher has already been configured for that new project name.

Before publishing, revoke any GitHub or PyPI tokens that were pasted into chats, issues, logs, or commits.

Pre-release checks

For release candidates that include runtime, connector, route, or CDC changes, build the explicit release checklist after the underlying gates have passed:

uv run dpone ops pre-release-checklist \
  --release vX.Y.Z \
  --release-type minor \
  --check cli_help_surface=true \
  --check cli_output_contracts=true \
  --check run_cli_manifest=true \
  --check run_python_api_manifest=true \
  --check nested_hierarchical_identity=true \
  --check nested_parent_child_integrity=true \
  --check source_sink_strategy_matrix=true \
  --check source_sink_artifacts=true \
  --check docker_live_routes=true \
  --check contracts_guardrails=true \
  --check documentation_yaml_examples=true \
  --check documentation_links=true \
  --check documentation_mkdocs=true \
  --check ci_cd_quality=true \
  --check package=true \
  --output-dir test_artifacts/release/vX.Y.Z/pre-release \
  --format json

uv sync --all-extras
uv run ruff check .
uv run ruff format --check .
uv run mypy --config-file mypy.ini
uv run pytest -m "not integration_live"
uv run dpone docs check-docs
uv run dpone docs check-airflow-public-contracts
uv run dpone docs update-airflow-public-contract-reference --check
uv run mkdocs build --strict
uv build
uv build packages/dpone-native-accel --out-dir dist
uv build packages/dpone-airflow-pack --out-dir dist
uv build packages/apache-airflow-providers-dpone --out-dir dist
rm -f dist/.gitignore
uv run python tools/pypi_release_smoke_dist.py \
  --dist-dir dist \
  --expected-version X.Y.Z \
  --inventory-only \
  --format json \
  > test_artifacts/release/vX.Y.Z/candidate-inventory.json
uv run python tools/agent_policy/package_archive_gate.py dist/*.whl dist/*.tar.gz
uv run twine check dist/*

Exact release identity and required-check evidence:

release_tag=vX.Y.Z
release_commit="$(git rev-list -n 1 "${release_tag}")"

uv run python tools/agent_policy/release_identity_gate.py \
  --root . \
  --tag "${release_tag}" \
  --commit-sha "${release_commit}" \
  --remote-ref origin/master \
  --output "test_artifacts/release/${release_tag}/release_identity.json"

uv run python tools/agent_policy/release_commit_gate.py \
  --repo PaulKov/dpone \
  --commit-sha "${release_commit}" \
  --policy .agents/policy/github-branch-protection.yml \
  --github-token-env GITHUB_TOKEN \
  --timeout-seconds 300 \
  --poll-interval-seconds 15 \
  > "test_artifacts/release/${release_tag}/exact_commit_checks.json"

The second command reads the credential only from the named environment variable. Never pass its value as a CLI argument. Both reports must name the same full commit SHA; a report for a branch head, another tag, an untrusted producer, a pending check, or a previous workflow attempt is not reusable. The identity gate reads every package version, internal dependency pin, and the changelog section from git show <commit>:<path>. Uncommitted or concurrent worktree files are never accepted as evidence for the frozen release commit. The exact-commit gate likewise loads .agents/policy/github-branch-protection.yml only from that frozen commit and records policy_sha256 in the JSON report; a local --policy override that selects different bytes is rejected.

Secret hygiene checks:

detect-secrets scan --force-use-all-plugins $(git ls-files)
git grep -I -n -E 'ghp_|github_pat_|pypi-|AKIA|PRIVATE KEY' -- ':!uv.lock'

The commands above must not print real credentials. If a credential appears in chat, terminal output, issues, logs, commits, or pull requests, rotate it before publishing and use PyPI Trusted Publishing instead of a long-lived PyPI token.

Package smoke from a fresh environment:

python -m venv /tmp/dpone-release-smoke
/tmp/dpone-release-smoke/bin/pip install -U pip
/tmp/dpone-release-smoke/bin/pip install dist/*.whl
/tmp/dpone-release-smoke/bin/pip install \
  --find-links dist \
  "dpone[full,accel]==X.Y.Z"
/tmp/dpone-release-smoke/bin/pip check
/tmp/dpone-release-smoke/bin/python tools/package_smoke.py --project-root . --dpone-cmd /tmp/dpone-release-smoke/bin/dpone

The inventory gate treats dist/ as a closed candidate set: exactly four distributions, each at the requested version, each with exactly one wheel and one .tar.gz sdist. Any duplicate build variant, .zip, stray file, directory, or symlink fails before archive inspection, attestation, upload, or publication. Its JSON report records the eight filenames and SHA-256 digests.

The archive gate reads only ZIP/tar member metadata and fails closed when a candidate contains repository-internal instructions, VCS state, Codex/agent configuration, or .env* files. The release workflow runs both this gate and the fresh four-wheel install before uploading, attesting, or publishing the immutable candidate set.

PyPI resolver visibility smoke:

uv run python tools/pypi_release_smoke_dist.py \
  --dist-dir dist \
  --expected-version X.Y.Z \
  --timeout-seconds 900 \
  --poll-interval-seconds 30 \
  --install-smoke \
  --dpone-install-extra accel \
  --format md

This check is stricter than upload success. It verifies the version-specific PyPI endpoint, the project JSON endpoint, the Simple Repository API endpoint used by installers, every candidate wheel and sdist filename/SHA-256, the exact four-package inventory, and a fresh resolver install. A visible version with different or yanked public bytes is a failure.

Before tagging a route-heavy minor release, use the Release RC collector to turn GitHub CLI PR exports into finalizer-ready release inputs:

uv run dpone ops release-rc-collect \
  --release v0.10.0 \
  --previous-release v0.9.0 \
  --package-version 0.10.0 \
  --base-branch codex/route-certify-release-automation \
  --head-branch codex/rc-integration-finalizer \
  --pull-request-json test_artifacts/release/v0.10.0/prs/pr-75.json \
  --pull-request-json test_artifacts/release/v0.10.0/prs/pr-76.json \
  --artifact route_release_finalizer=test_artifacts/route_release_finalize/route_release_finalizer.json \
  --artifact release_evidence_pack=test_artifacts/release/v0.10.0/release_evidence_pack.json \
  --format json

Then run the Release RC finalizer over the generated stacked PR train and release evidence:

uv run dpone ops release-rc-finalize \
  --release v0.10.0 \
  --previous-release v0.9.0 \
  --package-version 0.10.0 \
  --merge-train-json test_artifacts/release/v0.10.0/merge_train.json \
  --artifact route_release_finalizer=test_artifacts/route_release_finalize/route_release_finalizer.json \
  --artifact release_evidence_pack=test_artifacts/release/v0.10.0/release_evidence_pack.json \
  --format json

Do not create the tag while release_rc_finalizer.json is blocked. Run release-rc-finalize --mode post_merge after merging the train.

Release process

  1. Update CHANGELOG.md with a curated section for the target version.
  2. Ensure pyproject.toml contains the target version.
  3. Confirm the public names are available or owned by the maintainer: PaulKov/dpone on GitHub and dpone, dpone-native-accel, dpone-airflow-pack, and apache-airflow-providers-dpone on PyPI.
  4. Push changes to master through a reviewed PR.
  5. Wait for GitHub Actions CI to pass on master.
  6. Create an annotated tag:
git tag -a vX.Y.Z -m "Release vX.Y.Z"
git push origin vX.Y.Z
  1. The release preflight verifies the canonical annotated tag, exact workflow commit, ancestry from origin/master, all four package versions and pinned cross-package dependencies, the changelog section, frozen-source tenant hygiene via the CI-owned TENANT_HYGIENE_POLICY secret, and the live required GitHub checks including their configured producer identity.
  2. Separate least-privilege jobs build candidates (including archive-body tenant hygiene), attest them, recheck the annotated tag immediately before PyPI upload, select exactly one publisher, verify every public wheel and sdist byte, recheck the annotated tag again, and only then create the GitHub Release. Build and repository code never receive PYPI_API_TOKEN. A manual workflow dispatch must select the annotated tag; branch dispatches fail closed.
  3. The runtime image workflow independently rebuilds the exact candidate set and compares it with PyPI before building. A tag-push job publishes ghcr.io/paulkov/dpone-runtime:X.Y.Z, captures the registry digest, pulls and smokes that digest, emits an SPDX SBOM, and publishes provenance and SBOM attestations. Pull requests and manual dispatches are non-publishing.
  4. Verify the public install:
python -m venv /tmp/dpone-pypi-smoke
/tmp/dpone-pypi-smoke/bin/pip install -U pip
/tmp/dpone-pypi-smoke/bin/pip install "dpone[full]==X.Y.Z"
/tmp/dpone-pypi-smoke/bin/pip install "dpone[accel]==X.Y.Z"
/tmp/dpone-pypi-smoke/bin/pip install "apache-airflow-providers-dpone==X.Y.Z"
/tmp/dpone-pypi-smoke/bin/dpone --help

The release workflow runs tools/pypi_release_smoke_dist.py after upload and before creating the GitHub Release. It requires dpone, dpone-native-accel, dpone-airflow-pack, and apache-airflow-providers-dpone, all at the tag version, with one wheel and one sdist each. Public filenames, SHA-256 digests, yanked status, and the complete artifact set must match the locally built candidate. If PyPI accepts the upload but the installer path is stale or any byte differs, the workflow remains red and no GitHub Release is created.

Runtime image smoke:

image="ghcr.io/paulkov/dpone-runtime@sha256:<digest-from-runtime-image-evidence>"
docker run --rm "$image" --version
docker run --rm --entrypoint bcp "$image" -v
docker run --rm --entrypoint sqlcmd "$image" -?
docker run --rm --entrypoint clickhouse-client "$image" --version
docker run --rm "$image" runtime native-accel doctor --format json
docker run --rm "$image" airflow runtime-init-fetch --help
docker run --rm "$image" airflow runtime-pack-exec --help

Unified public artifact verification:

uvx --from "dpone==X.Y.Z" dpone ops release-verify \
  --release vX.Y.Z \
  --install-smoke \
  --timeout-seconds 1800 \
  --output test_artifacts/release/vX.Y.Z/release_verify.json \
  --format json

release-verify checks the GitHub Release tag, PyPI version endpoint, PyPI Simple Repository resolver visibility, optional fresh uvx install, and the ghcr.io/paulkov/dpone-runtime:X.Y.Z image manifest. Use the JSON output as a post-tag evidence artifact for CI/CD or manual release close.

For the full optional dependency bundle, pin a supported Python explicitly so local uvx does not select a newer interpreter than dpone supports:

uvx --from "dpone==X.Y.Z" dpone ops release-verify \
  --release vX.Y.Z \
  --install-smoke \
  --install-extra full \
  --install-python 3.11 \
  --timeout-seconds 1800 \
  --output test_artifacts/release/vX.Y.Z/release_verify_full.json \
  --format json

Publication blockers

Do not publish if any of these are true:

  • A GitHub or PyPI credential was pasted into chat, terminal output, docs, commits, issues, or pull requests and has not been rotated.
  • The release tag is lightweight, non-canonical, points to another commit, is not reachable from origin/master, or any package/changelog version differs.
  • GitHub Actions has not passed on the exact commit being tagged.
  • uv run twine check dist/* fails.
  • GitHub Artifact Attestation verification fails for any dist/*.whl or dist/*.tar.gz file, or the JSON verification receipt is missing.
  • tools/release_notes.py --version X.Y.Z cannot find a matching CHANGELOG.md section.
  • tools/pypi_release_smoke_dist.py --dist-dir dist --expected-version X.Y.Z cannot prove the complete four-package wheel/sdist candidate inventory and exact public SHA-256 identities.
  • The local candidate directory contains anything other than the exact eight expected artifacts, including duplicate variants, .zip, directories, symlinks, or unrelated files.
  • The runtime image workflow cannot build, smoke-test or publish ghcr.io/paulkov/dpone-runtime:X.Y.Z.
  • Runtime image evidence lacks the pushed digest, candidate checksums, installed package inventories, SPDX SBOM, provenance attestation, or digest-pinned post-push smoke.
  • dpone ops release-verify --release vX.Y.Z --install-smoke fails for the public GitHub, PyPI, or GHCR artifacts.
  • dpone ops release-verify --release vX.Y.Z --install-smoke --install-extra full --install-python 3.11 cannot install the full optional dependency bundle on a supported Python.
  • A fresh environment cannot install dpone, dpone[full], dpone[accel], and dpone-native-accel.
  • The package name dpone on PyPI or repo name PaulKov/dpone on GitHub is not available to the maintainer account.

Version policy

dpone uses SemVer once public APIs stabilize. During 0.x, minor versions may still include breaking changes, but they must be documented in CHANGELOG.md and Compatibility.