Skip to content

Manifest Checks: Unit Tests#

Note

The below checks require manifest.json to be present.

Checks related to unit test coverage and formats.

Functions:

Name Description
check_unit_test_coverage

Set the minimum percentage of models that have a unit test.
check_unit_test_expect_format

Unit tests can only use the specified formats.
check_unit_test_given_formats

Unit tests can only use the specified formats.

check_unit_test_coverage #

Set the minimum percentage of models that have a unit test.

Warning

This check is only supported for dbt 1.8.0 and above.

Parameters:

Name Type Description Default
min_unit_test_coverage_pct float

The minimum percentage of models that must have a unit test.

 100

Receives at execution time:

Name Type Description
models list[ModelNode]

List of ModelNode objects parsed from manifest.json.
unit_tests list[UnitTests]

List of UnitTests objects parsed from manifest.json.

Other Parameters (passed via config file):

Name Type Description
description str | None

Description of what the check does and why it is implemented.
include str | None

Regex pattern to match the model path. Only model paths that match the pattern will be checked.
severity Literal[error, warn] | None

Severity level of the check. Default: error.

Example(s): 

manifest_checks:
    - name: check_unit_test_coverage
      min_unit_test_coverage_pct: 90

Source code in src/dbt_bouncer/checks/manifest/check_unit_tests.py 
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
@check
def check_unit_test_coverage(
    ctx, *, include: str | None = None, min_unit_test_coverage_pct: int = 100
):
    """Set the minimum percentage of models that have a unit test.

    !!! warning

        This check is only supported for dbt 1.8.0 and above.

    Parameters:
        min_unit_test_coverage_pct (float): The minimum percentage of models that must have a unit test.

    Receives:
        models (list[ModelNode]): List of ModelNode objects parsed from `manifest.json`.
        unit_tests (list[UnitTests]): List of UnitTests objects parsed from `manifest.json`.

    Other Parameters:
        description (str | None): Description of what the check does and why it is implemented.
        include (str | None): Regex pattern to match the model path. Only model paths that match the pattern will be checked.
        severity (Literal["error", "warn"] | None): Severity level of the check. Default: `error`.

    Example(s):
        ```yaml
        manifest_checks:
            - name: check_unit_test_coverage
              min_unit_test_coverage_pct: 90
        ```

    """
    manifest_obj = ctx.manifest_obj
    if get_package_version_number(
        manifest_obj.manifest.metadata.dbt_version or "0.0.0"
    ) >= get_package_version_number("1.8.0"):
        relevant_models = [
            m.unique_id
            for m in ctx.models
            if object_in_path(include, m.original_file_path)
        ]
        models_with_unit_test = []
        for unit_test in ctx.unit_tests:
            if unit_test.depends_on and unit_test.depends_on.nodes:
                for node in unit_test.depends_on.nodes:
                    if node in relevant_models:
                        models_with_unit_test.append(node)

        num_models_with_unit_tests = len(set(models_with_unit_test))
        unit_test_coverage_pct = (
            num_models_with_unit_tests / len(relevant_models)
        ) * 100

        if unit_test_coverage_pct < min_unit_test_coverage_pct:
            fail(
                f"Only {unit_test_coverage_pct}% of models have a unit test, this is less than the permitted minimum of {min_unit_test_coverage_pct}%."
            )
    else:
        logging.warning(
            "The `check_unit_test_expect_format` check is only supported for dbt 1.8.0 and above."
        )

check_unit_test_expect_format #

Unit tests can only use the specified formats.

Warning

This check is only supported for dbt 1.8.0 and above.

Parameters:

Name Type Description Default
permitted_formats list[Literal[csv, dict, sql]] | None

A list of formats that are allowed to be used for expect input in a unit test.

 ['csv', 'dict', 'sql']

Receives at execution time:

Name Type Description
manifest_obj ManifestObject

The ManifestObject object parsed from manifest.json.
unit_test UnitTests

The UnitTests object to check.

Other Parameters (passed via config file):

Name Type Description
description str | None

Description of what the check does and why it is implemented.
exclude str | None

Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Unit test paths that match the pattern will not be checked.
include str | None

Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Only unit test paths that match the pattern will be checked.
severity Literal[error, warn] | None

Severity level of the check. Default: error.

Example(s): 

manifest_checks:
    - name: check_unit_test_expect_format
      permitted_formats:
        - csv

Source code in src/dbt_bouncer/checks/manifest/check_unit_tests.py 
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
@check
def check_unit_test_expect_format(
    unit_test,
    ctx,
    *,
    permitted_formats: list[str] = ["csv", "dict", "sql"],  # noqa: B006
):
    """Unit tests can only use the specified formats.

    !!! warning

        This check is only supported for dbt 1.8.0 and above.

    Parameters:
        permitted_formats (list[Literal["csv", "dict", "sql"]] | None): A list of formats that are allowed to be used for `expect` input in a unit test.

    Receives:
        manifest_obj (ManifestObject): The ManifestObject object parsed from `manifest.json`.
        unit_test (UnitTests): The UnitTests object to check.

    Other Parameters:
        description (str | None): Description of what the check does and why it is implemented.
        exclude (str | None): Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Unit test paths that match the pattern will not be checked.
        include (str | None): Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Only unit test paths that match the pattern will be checked.
        severity (Literal["error", "warn"] | None): Severity level of the check. Default: `error`.

    Example(s):
        ```yaml
        manifest_checks:
            - name: check_unit_test_expect_format
              permitted_formats:
                - csv
        ```

    """
    manifest_obj = ctx.manifest_obj
    if get_package_version_number(
        manifest_obj.manifest.metadata.dbt_version or "0.0.0"
    ) >= get_package_version_number("1.8.0"):
        if unit_test.expect.format is None:
            fail(
                f"Unit test `{unit_test.name}` does not have an `expect` format defined. "
                f"Permitted formats are: {permitted_formats}."
            )

        format_value = (
            unit_test.expect.format.value if unit_test.expect.format else None
        )

        if format_value not in permitted_formats:
            fail(
                f"Unit test `{unit_test.name}` has an `expect` format that is not permitted. "
                f"Permitted formats are: {permitted_formats}. "
                f"Found: {format_value}"
            )
    else:
        logging.warning(
            "The `check_unit_test_expect_format` check is only supported for dbt 1.8.0 and above."
        )

check_unit_test_given_formats #

Unit tests can only use the specified formats.

Warning

This check is only supported for dbt 1.8.0 and above.

Parameters:

Name Type Description Default
permitted_formats list[Literal[csv, dict, sql]] | None

A list of formats that are allowed to be used for expect input in a unit test.

 ['csv', 'dict', 'sql']

Receives at execution time:

Name Type Description
manifest_obj ManifestObject

The ManifestObject object parsed from manifest.json.
unit_test UnitTests

The UnitTests object to check.

Other Parameters (passed via config file):

Name Type Description
description str | None

Description of what the check does and why it is implemented.
exclude str | None

Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Unit test paths that match the pattern will not be checked.
include str | None

Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Only unit test paths that match the pattern will be checked.
severity Literal[error, warn] | None

Severity level of the check. Default: error.

Example(s): 

manifest_checks:
    - name: check_unit_test_given_formats
      permitted_formats:
        - csv

Source code in src/dbt_bouncer/checks/manifest/check_unit_tests.py 
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
@check
def check_unit_test_given_formats(
    unit_test,
    ctx,
    *,
    permitted_formats: list[str] = ["csv", "dict", "sql"],  # noqa: B006
):
    """Unit tests can only use the specified formats.

    !!! warning

        This check is only supported for dbt 1.8.0 and above.

    Parameters:
        permitted_formats (list[Literal["csv", "dict", "sql"]] | None): A list of formats that are allowed to be used for `expect` input in a unit test.

    Receives:
        manifest_obj (ManifestObject): The ManifestObject object parsed from `manifest.json`.
        unit_test (UnitTests): The UnitTests object to check.

    Other Parameters:
        description (str | None): Description of what the check does and why it is implemented.
        exclude (str | None): Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Unit test paths that match the pattern will not be checked.
        include (str | None): Regex pattern to match the unit test path (i.e the .yml file where the unit test is configured). Only unit test paths that match the pattern will be checked.
        severity (Literal["error", "warn"] | None): Severity level of the check. Default: `error`.

    Example(s):
        ```yaml
        manifest_checks:
            - name: check_unit_test_given_formats
              permitted_formats:
                - csv
        ```

    """
    manifest_obj = ctx.manifest_obj
    if get_package_version_number(
        manifest_obj.manifest.metadata.dbt_version or "0.0.0"
    ) >= get_package_version_number("1.8.0"):
        given_formats = [
            i.format.value for i in unit_test.given if i.format is not None
        ]
        if not all(e in permitted_formats for e in given_formats):
            fail(
                f"Unit test `{unit_test.name}` has given formats which are not permitted. Permitted formats are: {permitted_formats}."
            )
    else:
        logging.warning(
            "The `check_unit_test_given_formats` check is only supported for dbt 1.8.0 and above."
        )