Skip to content

Credentials quickstart

This page is the short version. Use it to choose a credential provider quickly, then move to the full Connections and credentials guide when you need connector-specific fields or runbooks.

Provider decision table

Provider Use when Good for
env Local development, CI smoke jobs, containers Clear copy-paste setup
params Tests, examples, generated manifests Self-contained demos
airflow Airflow DAG deployments Reusing Airflow Connections
vault Production secrets in HashiCorp Vault Centralized secret rotation

Environment variables

Use connection_type: env and a normalized connection id.

source:
  type: postgres
  connection_id: postgres_oltp
  connection_type: env
export DPONE_CONN_POSTGRES_OLTP_HOST=127.0.0.1
export DPONE_CONN_POSTGRES_OLTP_PORT=5432
export DPONE_CONN_POSTGRES_OLTP_DATABASE=app
export DPONE_CONN_POSTGRES_OLTP_USER=app
export DPONE_CONN_POSTGRES_OLTP_PASSWORD=secret

Inline params

Use params for demos, tests, generated examples, or local notebooks. Do not commit real secrets.

source:
  type: api
  api_type: rest
  connection_id: demo_api
  connection_type: params
  credentials:
    base_url: https://api.example.com
    token: "${DEMO_API_TOKEN}"

Airflow Connections

Use connection_type: airflow when dpone runs inside Airflow and should resolve connection ids from Airflow metadata.

sink:
  type: mssql
  connection_id: mssql_dwh
  connection_type: airflow

Vault

Use connection_type: vault when secrets are stored in HashiCorp Vault through vault-kv-client.

sink:
  type: clickhouse
  connection_id: clickhouse_dwh
  connection_type: vault
  vault_path: databases/clickhouse_dwh

Connector coverage

The shared credential flow supports all first-class source/sink families:

Family Env Params Airflow Vault
PostgreSQL yes yes yes yes
MSSQL yes yes yes yes
ClickHouse yes yes yes yes
BigQuery yes yes yes yes
Kafka yes yes yes yes
REST API yes yes yes yes

Next steps

  • Use the complete Connections and credentials guide for field names, examples, and troubleshooting.
  • Run dpone doctor --profile local before a first run.
  • Keep secrets out of manifests unless they are placeholders resolved by environment variables.