ADR 015: Data Governance Standards

Status: Proposed | Date: 2025-07-28

Context

Data transformation pipelines require basic governance to ensure quality and compliance. SQLMesh provides built-in capabilities for contracts and lineage that reduce governance overhead.

• SQLMesh Documentation
• Australian Government Data Governance Framework

Decision

Use SQLMesh’s semantic understanding for automated data governance with git-based workflows.

Priority Focus Areas

• Data Contracts: SQLMesh model definitions serve as schema contracts with automatic breaking change detection
• Column-Level Lineage: Automatic lineage tracking through SQLMesh semantic analysis
• Quality Validation: SQLMesh unit tests and data diff for transformation validation
• Audit Integration: Follow ADR 007: Centralized Security Logging for transformation logs

Consequences

Benefits

• Automated Governance: SQLMesh reduces manual governance overhead
• Git-Based Workflow: Familiar version control approach
• Built-in Validation: Immediate feedback on quality and contract violations

Trade-offs

• SQLMesh Dependency: Governance tied to SQLMesh framework
• Learning Curve: Teams need SQLMesh knowledge