staging/
# 1:1 with source, for light cleaning, renaming, and type casting. Provides a durable, idempotent base.
Best Practices and Project Structure
Modern Data Engineering
Separating data transformation logic is key to building trust and maintainability. In our dbt projects, we enforce a clear, three-tiered structure.
Best Practices and Project Structure
The principle of separating data transformation logic is key to building trust and maintainability. In our dbt projects, we enforce a clear, three-tiered structure.
models/
intermediate/
# Complex business logic is modularised here. Not exposed to end users.
marts/
# Final, aggregated models for consumption by BI tools. These are the "products" of the data team.
Our key principles for dbt:
Test Everything
We combine generic tests (e.g., `not_null`, `unique`) with singular, custom tests that encode critical business logic (e.g., `assert_total_revenue_is_positive`), ensuring data is not just present, but correct.
Document Everything
We enforce 100% documentation coverage for all models and columns, which feeds a living, searchable data catalogue for the entire organisation.
Favour Modularity
We break down complex business logic into smaller, reusable intermediate models. This improves readability, speeds up development, and makes testing more granular and effective.
Put our expertise to work
These frameworks are the starting point for our client engagements. If you're facing similar challenges, the next step is a complimentary discovery call.
Book a Discovery Call