Quality metrics

This page is the single source of truth for dpone code-quality metrics. It contains the current generated snapshot, interpretation rules, CI gates and update commands.

Current quality summary

🟡 YELLOW - architecture hard gates pass locally; clustering target and exact-cap legacy debt remain visible.

General import/layer checks do not replace the independent architecture gates. The clustering hard gate and stricter pre-release cross-layer assertion in tests/test_architecture_fitness_gate.py pass on the current source. The generated dashboard's heuristic OK labels do not establish release readiness or replace the complete exact-candidate CI suite. Regeneration updates observations, not acceptance thresholds; the preferred clustering target remains unmet.

Generic reconciliation/API/type facades are protected by import rules, broad facades are lazy/delegating, module-size hard violations are closed and legacy warning debt in the canonical src/dpone package is held by exact non-regression caps. The narrow architecture margin still warrants improvement. Release governance is split into focused catalog sections, so current architecture fitness has no class responsibility advisories. ETLProcessor is now below the preferred fan-out target after moving extracted-payload load-path coordination behind a focused service, MSSQL source strategies are below the preferred fan-out target after moving queryout artifact construction behind a focused factory, ClickHouse sink payload ingestion now lives behind a dedicated service, and PostgreSQL/BigQuery sink strategy wiring is isolated behind dedicated composition factories and shared connector/state ports. Strategy intelligence public exports are now lazy, so package-level compatibility no longer creates a false god-module dependency hotspot. Nested spill-to-disk loading now lives behind a focused service, keeping NestedLoadService as the root/child orchestrator.

The producer-owned current generated snapshot below is the numeric authority. Do not duplicate its exact metrics in this manual summary; CI checks that snapshot against the tracked repository tree.

Traffic-light interpretation: - 🟢 GREEN: no module-size failures, no import-rule violations, clustering is within target, and generated docs are current. - 🟡 YELLOW: no critical blocker, but exact-cap legacy size debt, advisory warning debt in an explicitly scoped no-baseline package, coupling debt, or a metric near its threshold remains visible. - 🔴 RED: forbidden imports, new unbaselined warning-level LOC/SLOC debt, growth above an exact cap, any hard-limit violation, stale generated docs, or a worsening layer-metrics regression.

First recommendations: - Keep newly touched modules at or below both warning thresholds: 450 LOC and 350 SLOC. New warning-level debt fails the baseline-mode gate; the absolute hard limits remain 600 LOC and 400 SLOC. - Keep command/runtime code behind service/protocol facades; do not import vendor adapters from generic core. - Keep small route/control-plane dependency triangles visible before adding the next broad route feature. - Keep ReleaseOpsCatalog as a composition root only; add new release behavior to focused gate/governance/readiness/risk sections. - Treat dpone.manifest.explain as the next architecture-fitness fan-out reduction target; benchmark scoring also keeps dpone.runtime.credentials.factory visible in the production-only corpus. - Treat broad facades as compatibility shims only: lazy re-export/delegation is acceptable, business logic is not.

Current generated snapshot

This section is auto-generated by dpone docs update-dev-metrics.

Quality summary

• Architecture graph/hard-gate status: 🟡 warning
• Repository-wide traffic light also includes exact-cap legacy debt; see the manual summary above.
• Biggest module: 449 LOC (src/dpone/commands/airflow_self_service_rendering.py) → OK
• Avg clustering: 0.182 → WARN
• Cross-layer ratio: 0.299 → OK
• First improvement target: keep broad service/command seams thin before adding new runtime features.

LOC (lines of code)

Repository-wide (**/*.py)

• Files: 6094
• Total lines: 1185390
• Total SLOC: 1015160
• Min: 0 (src/dpone/adapters/__init__.py)
• Max: 4153 (tests/test_airflow_self_service_cli.py)
• Avg: 194.52
• Median: 145

Repository-wide without tests (**/*.py, excluding tests/)

• Files: 4281
• Total lines: 681649
• Total SLOC: 585741
• Max: 1043 (tools/type_matrix_certification.py)
• Avg: 159.23
• Median: 129

Top 15 largest modules in src/dpone/

Module	Lines
src/dpone/commands/airflow_self_service_rendering.py	449
src/dpone/services/safe_sample_policy.py	449
src/dpone/security_redaction.py	446
src/dpone/manifest/confined_mutations.py	443
src/dpone/runtime/connectors/api/connector.py	436
src/dpone/gitops/airflow_dag_spec.py	434
src/dpone/runtime/connectors/api/omnidesk_connector.py	431
src/dpone/runtime/credentials/binding_resolver.py	429
src/dpone/commands/dbt_promotion_cmd.py	427
src/dpone/gitops/airflow_connection_bridge_plan.py	426
src/dpone/contracts/dbt_sqlserver_graph_rules.py	424
src/dpone/services/schema_type_matrix.py	423
src/dpone/runtime/native_transfer.py	422
src/dpone/runtime/runtime_init_fetch_ready.py	421
src/dpone/runtime/sources/strategies/clickhouse/clickhouse_incremental_extract.py	421

Coupling / Cohesion (internal imports inside dpone.*)

• Modules: 3713
• Internal import edges: 9565
• AVG Ce (fan-out): 2.58
• Median Ce: 2
• P90 Ce: 6
• P95 Ce: 8
• MAX Ce: 24 (dpone.app.dbt_promotion_composition)
• MAX Ca (fan-in): 102 (dpone.commands.output_json)
• LCC ratio (largest connected component): 0.948
• Avg clustering coefficient: 0.182
• Cohesion ratio (share of deps within slice): 0.618

Heuristic interpretation

• AVG Ce = 2.58 → OK (lower is usually better; big fan-out hints 'god modules')
• P90 Ce = 6 → OK (if P90 grows, responsibilities are spreading)
• Avg clustering = 0.182 → WARN (higher clustering often means more cycles/tight coupling)

Top 15 modules by fan-out (Ce)

• dpone.app.dbt_promotion_composition: 24
• dpone.gitops.schema_airflow_contracts: 24
• dpone.app.dbt_publish_composition: 22
• dpone.app.semantic_refresh_composition: 22
• dpone.runtime.sources.strategies.mssql.mssql_queryout_artifacts: 21
• dpone.app.dbt_workspace_composition: 20
• dpone.commands.airflow_self_service_cmd: 20
• dpone.readiness.managed_planning: 20
• dpone.app.semantic_refresh_worker_composition: 19
• dpone.gitops.schema_contracts: 19
• dpone.runtime.sinks.strategies.mssql.mssql_strategy_base: 19
• dpone.runtime.sources.strategies.postgres.postgres_xmin_extract: 19
• dpone.app.semantic_refresh_recovery_composition: 18
• dpone.readiness.airflow_self_service_application: 18
• dpone.runtime.etl.mssql_transaction_admission: 18

Top 15 modules by fan-in (Ca)

Approved stable high fan-in contracts are expected shared DTO/port modules; they remain visible here but are treated differently from god-module risks.

• dpone.commands.output_json: 102 - approved stable contract: output serialization port
• dpone.commands.output_text: 99 - approved stable contract: output text rendering port
• dpone.contracts.airflow_deployment: 91
• dpone.manifest.confined_files: 77
• dpone.readiness.migration_control: 76
• dpone.gitops.models: 60
• dpone.contracts.dbt_contract_validation: 55
• dpone.gitops.schema_contract_primitives: 55
• dpone.commands.func_command: 54
• dpone.runtime.sources.extract_result: 50 - approved stable contract: source extraction DTO
• dpone.runtime.sinks.load_result: 47 - approved stable contract: sink load result DTO
• dpone.ops.routes.models: 46
• dpone.runtime.artifact_models: 45
• dpone.runtime.deployment_cache_common: 45
• dpone.config.load_strategy: 44 - approved stable contract: load strategy enum contract

Layer / Slice architecture metrics

• Layers observed: 58
• Internal import edges: 9426
• Intra-layer edges: 6608
• Cross-layer edges: 2818
• Intra-layer ratio: 0.701

• Cross-layer ratio: 0.299

• Excluded layers: dpone.compat (150 modules, 139 edges removed from this coarse architecture view)

Heuristic interpretation

• Intra-layer ratio = 0.701 → OK (higher means more dependencies stay within their own layer)
• Cross-layer ratio = 0.299 → OK (lower means cleaner boundaries between layers)
• Top cross-layer / cross-slice flows below help spot architecture seams that may need ports, facades or further extraction.

Layer summary

Layer	Modules	Out	In	Intra	Cross-out	Cross-in	Avg Ce	Avg Ca	Cohesion	Target layers	Source layers
dpone._compat	1	0	32	0	0	32	0.00	32.00	1.000	0	13
dpone.adapters	187	413	268	178	235	90	2.21	1.43	0.431	5	5
dpone.airflow	8	0	0	0	0	0	0.00	0.00	1.000	0	0
dpone.airflow_connection_names	1	0	7	0	0	7	0.00	7.00	1.000	0	3
dpone.api	1	6	0	0	6	0	6.00	0.00	0.000	2	0
dpone.app	23	188	33	15	173	18	8.17	1.43	0.080	10	4
dpone.backfill	73	168	207	139	29	68	2.30	2.84	0.827	6	5
dpone.cli	4	9	4	3	6	1	2.25	1.00	0.333	4	1
dpone.cli_render	24	46	43	19	27	24	1.92	1.79	0.413	8	3
dpone.commands	238	922	583	581	341	2	3.87	2.45	0.630	22	1
dpone.config	16	33	115	18	15	97	2.06	7.19	0.545	3	10
dpone.connector_sdk	12	9	13	9	0	4	0.75	1.08	1.000	0	1
dpone.contracts	283	668	1463	657	11	806	2.36	5.17	0.984	5	18
dpone.core	7	8	1	1	7	0	1.14	0.14	0.125	4	0
dpone.dag	40	107	115	72	35	43	2.67	2.88	0.673	5	6
dpone.gitops	201	505	587	418	87	169	2.51	2.92	0.828	10	6
dpone.governance	6	6	31	5	1	26	1.00	5.17	0.833	1	5
dpone.integration_matrix	1	2	3	0	2	3	2.00	3.00	0.000	2	2
dpone.integration_matrix_behavior	1	1	3	0	1	3	1.00	3.00	0.000	1	3
dpone.integration_matrix_constants	1	0	6	0	0	6	0.00	6.00	1.000	0	6
dpone.integration_matrix_counts	1	1	5	0	1	5	1.00	5.00	0.000	1	5
dpone.integration_matrix_manifest	1	1	1	0	1	1	1.00	1.00	0.000	1	1
dpone.integration_matrix_models	1	5	1	0	5	1	5.00	1.00	0.000	5	1
dpone.integration_matrix_samples	1	4	1	0	4	1	4.00	1.00	0.000	4	1
dpone.integration_matrix_selection	1	1	1	0	1	1	1.00	1.00	0.000	1	1
dpone.integration_matrix_simulation	1	4	1	0	4	1	4.00	1.00	0.000	4	1
dpone.integration_matrix_wide	1	1	1	0	1	1	1.00	1.00	0.000	1	1
dpone.kubernetes_names	1	0	10	0	0	10	0.00	10.00	1.000	0	4
dpone.lazy_exports	1	0	5	0	0	5	0.00	5.00	1.000	0	3
dpone.lib	14	25	11	10	15	1	1.79	0.79	0.400	3	1
dpone.load_profile	5	8	7	5	3	2	1.60	1.40	0.625	2	1
dpone.manifest	122	374	540	312	62	228	3.07	4.43	0.834	7	13
dpone.metrics	23	32	43	28	4	15	1.39	1.87	0.875	1	1
dpone.observability	7	2	3	2	0	1	0.29	0.43	1.000	0	1
dpone.operations	38	38	0	0	38	0	1.00	0.00	0.000	1	0
dpone.ops	277	635	593	528	107	65	2.29	2.14	0.831	13	4
dpone.orchestration	6	6	9	5	1	4	1.00	1.50	0.833	1	1
dpone.output	1	5	7	0	5	7	5.00	7.00	0.000	5	1
dpone.output_files	1	0	2	0	0	2	0.00	2.00	1.000	0	2
dpone.output_format	1	1	1	0	1	1	1.00	1.00	0.000	1	1
dpone.output_json	1	1	4	0	1	4	1.00	4.00	0.000	1	3
dpone.output_table	1	0	11	0	0	11	0.00	11.00	1.000	0	2
dpone.output_text	1	1	3	0	1	3	1.00	3.00	0.000	1	3
dpone.output_yaml	1	0	2	0	0	2	0.00	2.00	1.000	0	2
dpone.ports	102	104	342	52	52	290	1.02	3.35	0.500	3	12
dpone.readiness	382	1009	862	628	381	234	2.64	2.26	0.622	21	7
dpone.root	1	1	0	0	1	0	1.00	0.00	0.000	1	0
dpone.runtime	1030	3058	2698	2498	560	200	2.97	2.62	0.817	17	10
dpone.schema	1	1	1	0	1	1	1.00	1.00	0.000	1	1
dpone.security_redaction	1	0	27	0	0	27	0.00	27.00	1.000	0	9
dpone.services	328	916	543	357	559	186	2.79	1.66	0.390	20	9
dpone.staging	2	4	1	1	3	0	2.00	0.50	0.250	2	0
dpone.storage	8	10	39	8	2	31	1.25	4.88	0.800	1	5
dpone.strategy_intelligence	38	55	49	32	23	17	1.45	1.29	0.582	7	5
dpone.supply_chain	8	5	5	3	2	2	0.62	0.62	0.600	2	1
dpone.type_system	24	27	72	24	3	48	1.12	3.00	0.889	3	5
dpone.vault_references	1	0	4	0	0	4	0.00	4.00	1.000	0	4
dpone.version	1	0	7	0	0	7	0.00	7.00	1.000	0	5

Top 15 cross-layer dependency flows

From layer	To layer	Edges
dpone.runtime	dpone.contracts	214
dpone.services	dpone.contracts	155
dpone.adapters	dpone.contracts	119
dpone.adapters	dpone.ports	111
dpone.services	dpone.gitops	109
dpone.commands	dpone.services	100
dpone.services	dpone.ports	100
dpone.readiness	dpone.manifest	96
dpone.readiness	dpone.runtime	83
dpone.commands	dpone.readiness	81
dpone.readiness	dpone.contracts	79
dpone.services	dpone.readiness	67
dpone.runtime	dpone.config	65
dpone.ops	dpone.runtime	59
dpone.runtime	dpone.backfill	57

Top 15 cross-slice dependency flows

From slice	To slice	Edges
dpone.runtime	dpone.contracts	214
dpone.adapters	dpone.contracts	119
dpone.adapters	dpone.ports	111
dpone.services.gitops	dpone.gitops	102
dpone.readiness	dpone.manifest	96
dpone.readiness	dpone.runtime	83
dpone.readiness	dpone.contracts	79
dpone.runtime	dpone.config	65
dpone.ops	dpone.runtime	59
dpone.runtime	dpone.backfill	57
dpone.runtime	dpone.readiness	55
dpone.app	dpone.adapters	52
dpone.ports	dpone.contracts	50
dpone.runtime	dpone.ports	49
dpone.manifest	dpone.contracts	46

How to update this page

dpone docs update-dev-metrics

CI check:

dpone docs update-dev-metrics --check

Layer-metrics gate (thresholds + regression vs baseline):

dpone docs check-layer-metrics

Architecture-fitness gate (coupling hot spots + high-responsibility classes):

dpone docs check-architecture-fitness

Module-size gate (LOC/SLOC + explicit debt baseline):

HEAD_SHA="$(git rev-parse HEAD)"
BASE_SHA="$(git merge-base "$HEAD_SHA" origin/master)"
if [ "$BASE_SHA" = "$HEAD_SHA" ]; then BASE_SHA="$(git rev-parse "${HEAD_SHA}^")"; fi
dpone docs check-module-size \
  --baseline docs/module_size_baseline.json \
  --base-ref "$BASE_SHA" \
  --head-ref "$HEAD_SHA"

Refresh the committed baseline snapshot:

dpone docs check-layer-metrics --write-baseline
HEAD_SHA="$(git rev-parse HEAD)"
BASE_SHA="$(git merge-base "$HEAD_SHA" origin/master)"
if [ "$BASE_SHA" = "$HEAD_SHA" ]; then BASE_SHA="$(git rev-parse "${HEAD_SHA}^")"; fi
dpone docs check-module-size \
  --baseline docs/module_size_baseline.json \
  --base-ref "$BASE_SHA" \
  --head-ref "$HEAD_SHA" \
  --write-baseline

The module-size writer requires a committed refactor, returns exit 2 after a successful candidate write, and needs a second exact-head check after the baseline is committed. See the module-size ratchet runbook.

Metric groups

1. LOC / SLOC (lines of code)

These metrics show raw module size and help spot oversized files. LOC counts all physical lines; SLOC counts non-empty, non-comment source lines. The generated report tracks both repository-wide metrics and the same metrics excluding tests, because production module size and test volume have different meanings.

Useful heuristics: - max LOC < 600 is the hard target for “regular” modules. - LOC <= 450 is the warning target for newly touched modules. - files above ~800-1000 LOC are strong candidates for splitting. - median LOC is often more informative than average, because a few large modules can skew the average.

2. Module-level coupling / cohesion

These metrics are based on the internal import graph inside dpone.*.

• Ce (fan-out) — how many internal modules a module imports. Lower is usually better.
• Ca (fan-in) — how many internal modules import a module. High fan-in can be fine for stable shared abstractions.
• LCC ratio — size of the largest connected component in the import graph. Very high values can mean the system is too entangled.
• Avg clustering — a proxy for “triangles” in the graph. Higher values often indicate tight coupling and hidden cycles.
• Cohesion ratio — share of imports that stay within the same slice. Higher is usually better.

Heuristics used in the generated report: - AVG Ce: OK ≤ 6, WARN ≤ 12, ALERT > 12 - P90 Ce: OK ≤ 12, WARN ≤ 18, ALERT > 18 - Avg clustering: OK ≤ 0.18, WARN ≤ 0.25, ALERT > 0.25

3. Layer / slice architecture metrics

These metrics complement module-level coupling by looking at architectural boundaries.

• Layer summary groups modules into coarse layers such as dpone.commands, dpone.services, dpone.manifest, dpone.dag, dpone.runtime, dpone.ports, dpone.adapters.
• Cross-layer flows show the strongest allowed layer→layer dependencies.
• Cross-slice flows go one level deeper (commands.dag -> services.dag, services.docs -> metrics, etc.) and help identify where abstractions may still leak.

Useful heuristics: - Intra-layer ratio — higher is better. A rough target is >= 0.55. - Cross-layer ratio — lower is better. A rough target is <= 0.45.

These are not absolute pass/fail thresholds. Some orchestration-heavy layers (commands, services) will naturally depend on several other layers. The main value is tracking trends over time and spotting unexpectedly heavy flows.

Baseline vs target

The current generated snapshot is recorded above on this page. Regression baselines live in docs/layer_metrics_baseline.json and docs/module_size_baseline.json.

Recommended long-term goals: - max module LOC < 600 - max module SLOC <= 400 - max Ce < 12 - avg clustering <= 0.18 - no forbidden cross-layer imports (enforced by dpone docs check-import-rules) - no new warning-level debt or growth above exact baseline caps (enforced by dpone docs check-module-size) - cross-layer ratio trending downward release over release

Layer-metrics gate in CI

Recommended CI step:

dpone docs check-layer-metrics \
  --baseline docs/layer_metrics_baseline.json \
  --min-intra-layer-ratio 0.55 \
  --max-cross-layer-ratio 0.45 \
  --allowed-ratio-regression 0.02 \
  --allowed-flow-regression 5

What it checks: - absolute thresholds for intra-layer / cross-layer ratios; - optional cap for the largest cross-layer flow; - regression against the committed baseline snapshot in docs/layer_metrics_baseline.json.

This is intentionally a coarse architectural gate. It does not replace dpone docs check-import-rules; instead, it complements it by catching trend regressions even when no forbidden import rule is violated.

Module-size gate in CI

Recommended CI step:

dpone docs check-module-size \
  --baseline docs/module_size_baseline.json \
  --base-ref "<full-base-commit-sha>" \
  --head-ref "<full-checked-out-head-sha>" \
  --warn-lines 450 \
  --max-lines 600 \
  --warn-sloc 350 \
  --max-sloc 400

What it checks: - LOC and SLOC per Python module; - absolute LOC/SLOC hard limits even for baselined modules; - unbaselined warning-level debt and growth above exact recorded caps; - stale/deleted entries, target-date expiry, Git ancestry and rename/split continuity; - focused refactor progress that must lower or remove its checked-in cap.

This is the authoritative baseline-mode contract for src/dpone. Separate --package ... --no-baseline invocations are scoped hard-budget checks: global hard-limit failures remain blocking, while warning findings are advisory and the report declares debt_model: none. The CLI rejects no-baseline mode for the canonical package, so it cannot bypass this ledger.

Current baseline policy: - No baseline entry can waive 600 LOC or 400 SLOC. - Caps equal the measured debt exactly; hidden headroom is invalid. - New baseline entries require an owner, reason, target/date, full baseline commit and Accepted ADR. - Compatibility facades are allowed, but must delegate immediately and must not accumulate business logic.