Credentials quickstart¶
This page is the short version. Use it to choose a credential provider quickly, then move to the full Connections and credentials guide when you need connector-specific fields or runbooks.
Provider decision table¶
| Provider | Use when | Good for |
|---|---|---|
env |
Local development, CI smoke jobs, containers | Clear copy-paste setup |
params |
Tests, examples, generated manifests | Self-contained demos |
airflow |
Airflow DAG deployments | Reusing Airflow Connections |
vault |
Production secrets in HashiCorp Vault | Centralized secret rotation |
Environment variables¶
Use connection_type: env and a normalized connection id.
export DPONE_CONN_POSTGRES_OLTP_HOST=127.0.0.1
export DPONE_CONN_POSTGRES_OLTP_PORT=5432
export DPONE_CONN_POSTGRES_OLTP_DATABASE=app
export DPONE_CONN_POSTGRES_OLTP_USER=app
export DPONE_CONN_POSTGRES_OLTP_PASSWORD=secret
Inline params¶
Use params for demos, tests, generated examples, or local notebooks. Do not commit real secrets.
source:
type: api
api_type: rest
connection_id: demo_api
connection_type: params
credentials:
base_url: https://api.example.com
token: "${DEMO_API_TOKEN}"
Airflow Connections¶
Use connection_type: airflow when dpone runs inside Airflow and should resolve connection ids from Airflow metadata.
Vault¶
Use connection_type: vault when secrets are stored in HashiCorp Vault through vault-kv-client.
sink:
type: clickhouse
connection_id: clickhouse_dwh
connection_type: vault
vault_path: databases/clickhouse_dwh
Connector coverage¶
The shared credential flow supports all first-class source/sink families:
| Family | Env | Params | Airflow | Vault |
|---|---|---|---|---|
| PostgreSQL | yes | yes | yes | yes |
| MSSQL | yes | yes | yes | yes |
| ClickHouse | yes | yes | yes | yes |
| BigQuery | yes | yes | yes | yes |
| Kafka | yes | yes | yes | yes |
| REST API | yes | yes | yes | yes |
Next steps¶
- Use the complete Connections and credentials guide for field names, examples, and troubleshooting.
- Run
dpone doctor --profile localbefore a first run. - Keep secrets out of manifests unless they are placeholders resolved by environment variables.