Skip to content

Managed-like UX for dpone

dpone includes self-service commands that help users diagnose their environment, generate manifests, preview execution plans, inspect state, write run artifacts, certify connectors, tune performance, and scaffold community connectors.

Fast path

dpone doctor --profile local
dpone init \
  --source-type postgres \
  --sink-type mssql \
  --source-connection pg_src \
  --sink-connection mssql_dwh \
  --source-schema public \
  --source-table orders \
  --target-schema landing \
  --target-table orders \
  --strategy incremental_merge \
  --unique-key id \
  --out examples/batch/landing_orders.batch.yaml
dpone plan examples/batch/landing_orders.batch.yaml --selector public.orders
dpone perf advise examples/batch/landing_orders.batch.yaml --selector public.orders

Commands

  • dpone doctor checks local readiness: Python, optional extras, Docker, bcp, sqlcmd, ClickHouse tools, Kafka/Schema Registry hints, and fix commands.
  • dpone init generates a batch manifest bundle, .env.example, and smoke command.
  • dpone plan renders a dry-run execution plan without changing targets.
  • dpone run-report writes JSON, Markdown, and HTML run artifacts under .dpone/runs by default.
  • dpone state exposes safe inspect/reset/export/replay/compare UX. Destructive operations preview unless --yes is supplied.
  • dpone connectors lists, certifies, and scaffolds connector SDK skeletons.
  • dpone perf advise recommends native bulk paths, partitioning, and delivery tuning.
  • dpone studio prints local Studio API metadata by default; --serve starts the headless local HTTP API bridge for the separate Nuxt/Vue dpone-studio UI.

Manifest additions

All managed UX sections are optional and backward-compatible:

quality:
  checks:
    - type: min_rows
      value: 1
      mode: warn

observability:
  artifacts:
    enabled: true
    formats: [json, md, html]
  opentelemetry:
    enabled: false
  prometheus:
    enabled: false

performance:
  advisor:
    enabled: true

certification:
  connector_status: experimental

Safety defaults

  • dpone plan is dry-run only.
  • dpone state reset and dpone state replay-from require --yes to execute; without it they return a preview payload.
  • Secrets are redacted in doctor, plan, run artifact, certification, and scaffold diagnostics.
  • Studio is local-only in this version and has no SaaS backend.

Studio architecture

dpone intentionally does not embed a SaaS/product frontend. The framework repo owns the headless local API bridge, while the UI lives in a separate Nuxt/Vue repo:

  • framework/API: dpone studio --serve;
  • frontend repo: PaulKov/dpone-studio;
  • contract: JSON endpoints listed below.

This keeps the ETL framework, connector runtime and frontend product independently deployable.

Studio API endpoints

Start the local Studio API bridge:

dpone studio --serve --host 127.0.0.1 --port 8765
  • / returns a minimal API status page and points users to the separate Nuxt/Vue Studio repo.
  • /api/studio returns Studio navigation, panels, quick actions and endpoint metadata for the frontend.
  • /api/doctor returns redacted environment diagnostics.
  • /api/connectors returns local connector/feature metadata.
  • /api/perf returns the same service-backed metadata envelope used by the performance panel.

Run artifacts

A run artifact contains timeline, state before/after, quality results, schema evolution notes, reconciliation metrics, and redacted diagnostics. Use JSON for automation, Markdown for repo evidence, and HTML for local review.

Connector SDK

dpone connectors scaffold demo_api --root ./community-connectors

The scaffold includes connector/source/sink skeletons, a contract test, and a docs template. Fill in auth, capabilities, examples, and certification evidence before publishing a community connector.

Production checklist

  • Run dpone doctor --profile production on the execution host.
  • Run dpone plan and inspect staging/shadow/schema/state sections.
  • Run dpone perf advise for large tables or Kafka batches.
  • Configure quality.checks for correctness gates.
  • Store run artifacts for every release or live certification run.