Manifest Checks: Lineage#
Note
The below checks require manifest.json to be present.
Functions:
| Name | Description |
|---|---|
check_lineage_permitted_upstream_models |
Upstream models must have a path that matches the provided |
check_lineage_permitted_upstream_sources |
Upstream sources must resolve to a database and/or schema that matches the provided patterns. |
check_lineage_seed_cannot_be_used |
Seed cannot be referenced in models with a path that matches the specified |
check_lineage_source_cannot_be_used |
Sources cannot be referenced in models with a path that matches the specified |
check_lineage_permitted_upstream_models
#
Upstream models must have a path that matches the provided upstream_path_pattern.
Rationale
A well-structured dbt project enforces clear layer boundaries — e.g. staging models only reference sources, intermediate models only reference staging or other intermediates, and marts only reference intermediates. Without this check, developers can inadvertently create cross-layer dependencies (a mart model directly referencing a staging model) that erode the project's modularity and make refactoring risky.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
upstream_path_pattern
|
str
|
Regexp pattern to match the upstream model(s) path. |
required |
Receives at execution time:
| Name | Type | Description |
|---|---|---|
manifest_obj |
ManifestObject
|
The manifest object. |
model |
ModelNode
|
The ModelNode object to check. |
models |
list[ModelNode]
|
List of ModelNode objects parsed from |
Other Parameters (passed via config file):
| Name | Type | Description |
|---|---|---|
description |
str | None
|
Description of what the check does and why it is implemented. |
exclude |
str | list[str] | None
|
Regex pattern(s) to match the model path. Model paths that match any pattern will not be checked. |
include |
str | list[str] | None
|
Regex pattern(s) to match the model path. Only model paths that match any pattern will be checked. |
severity |
Literal[error, warn] | None
|
Severity level of the check. Default: |
Example(s):
manifest_checks:
- name: check_lineage_permitted_upstream_models
include: ^models/staging
upstream_path_pattern: $^
- name: check_lineage_permitted_upstream_models
include: ^models/intermediate
upstream_path_pattern: ^models/staging|^models/intermediate
- name: check_lineage_permitted_upstream_models
include: ^models/marts
upstream_path_pattern: ^models/staging|^models/intermediate
Source code in src/dbt_bouncer/checks/manifest/check_lineage.py
check_lineage_permitted_upstream_sources
#
Upstream sources must resolve to a database and/or schema that matches the provided patterns.
Rationale
In a multi-project (dbt mesh) setup, one project often reads the output of another project through sources. Each layer must only read from the layers of other projects that it is permitted to use, e.g. a marts model must only read from the marts layer of an upstream domain. The source path cannot express this, but the resolved database and schema of the source can.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
database_pattern
|
str | None
|
Regexp pattern to match the resolved database of the upstream source(s). |
None
|
schema_pattern
|
str | None
|
Regexp pattern to match the resolved schema of the upstream source(s). |
None
|
Receives at execution time:
| Name | Type | Description |
|---|---|---|
model |
ModelNode
|
The ModelNode object to check. |
sources |
list[SourceNode]
|
List of SourceNode objects parsed from |
Other Parameters (passed via config file):
| Name | Type | Description |
|---|---|---|
description |
str | None
|
Description of what the check does and why it is implemented. |
exclude |
str | list[str] | None
|
Regex pattern(s) to match the model path. Model paths that match any pattern will not be checked. |
include |
str | list[str] | None
|
Regex pattern(s) to match the model path. Only model paths that match any pattern will be checked. |
severity |
Literal[error, warn] | None
|
Severity level of the check. Default: |
Note
At least one of database_pattern or schema_pattern must be supplied. If both are supplied, an upstream source must match both. A source with no database (e.g. on adapters without a database concept) does not match database_pattern.
Example(s):
manifest_checks:
- name: check_lineage_permitted_upstream_sources
include: ^models/marts|^models/consumption
database_pattern: ^dmn
schema_pattern: ^domain__
Source code in src/dbt_bouncer/checks/manifest/check_lineage.py
check_lineage_seed_cannot_be_used
#
Seed cannot be referenced in models with a path that matches the specified include config.
Rationale
Seeds are designed for small, static reference data (e.g. country codes, status mappings). Referencing seeds in intermediate or mart layers can indicate that data which should come from a source or staging model is instead being managed as a CSV file, making it harder to audit, version, and scale.
Receives at execution time:
| Name | Type | Description |
|---|---|---|
model |
ModelNode
|
The ModelNode object to check. |
Other Parameters (passed via config file):
| Name | Type | Description |
|---|---|---|
description |
str | None
|
Description of what the check does and why it is implemented. |
exclude |
str | list[str] | None
|
Regex pattern(s) to match the model path. Model paths that match any pattern will not be checked. |
include |
str | list[str] | None
|
Regex pattern(s) to match the model path. Only model paths that match any pattern will be checked. |
severity |
Literal[error, warn] | None
|
Severity level of the check. Default: |
Example(s):
manifest_checks:
- name: check_lineage_seed_cannot_be_used
include: ^models/intermediate|^models/marts
Source code in src/dbt_bouncer/checks/manifest/check_lineage.py
check_lineage_source_cannot_be_used
#
Sources cannot be referenced in models with a path that matches the specified include config.
Rationale
In a well-layered dbt project, raw sources should only be referenced from staging models. Allowing intermediate or mart models to query sources directly bypasses the staging layer, leads to duplicated transformation logic, and makes it harder to swap or rename sources without cascading changes across the project.
Receives at execution time:
| Name | Type | Description |
|---|---|---|
model |
ModelNode
|
The ModelNode object to check. |
Other Parameters (passed via config file):
| Name | Type | Description |
|---|---|---|
description |
str | None
|
Description of what the check does and why it is implemented. |
exclude |
str | list[str] | None
|
Regex pattern(s) to match the model path. Model paths that match any pattern will not be checked. |
include |
str | list[str] | None
|
Regex pattern(s) to match the model path. Only model paths that match any pattern will be checked. |
severity |
Literal[error, warn] | None
|
Severity level of the check. Default: |
Example(s):
manifest_checks:
- name: check_lineage_source_cannot_be_used
include: ^models/intermediate|^models/marts