Skip to main content

Unit tests for assets and ops

Unit testing is an important tool for verifying that a computation works as intended.

While this is traditionally difficult in the context of data pipelines, Dagster helps simplify this process by allowing you to directly invoke your computations with specific input values and mocked resources to ensure that they transform data as expected.

Unit tests can't fully replace integration tests or manual review, but are capable of catching a large variety of errors with a significantly faster feedback loop.

This guide covers how to write unit tests for assets and ops with a variety of different input requirements.

info

Unit testing individual assets or ops is generally recommended over unit testing entire jobs.

info

Unit testing isn't recommended in cases where most of the business logic is encoded in an external system, such as an asset which directly invokes an external Databricks job.

Prerequisites

Testing assets and ops with no arguments​

The simplest assets and ops to test are those with no arguments. In these cases, you can directly invoke your definition.

Loading...

Testing assets and ops that have upstream dependencies​

If your asset or op has an upstream dependency, then you can directly pass a value for that dependency when invoking your definition.

Loading...

Testing assets and ops with config​

If your asset or op uses config, you can construct an instance of the required config object and pass it in directly.

Loading...

Testing assets and ops with resources​

If your asset or op uses a resource, it can be useful to create a mock instance of that resource to avoid interacting with external services.

Loading...

Testing assets and ops with context​

If your asset or op uses a context argument, you can use build_asset_context() or build_op_context() to construct a context object.

Loading...

Testing assets and ops with a combination of parameters​

If your asset or op has multiple parameters, it's recommended to use keyword arguments for clarity.

Loading...

Next steps​