This repository serves as a demonstration of how to use Databricks Connect with serverless Spark to run unit and integration tests. The setup includes a Continuous Integration (CI) pipeline configured with GitHub Actions, which installs dependencies, sets up a Python environment, and runs tests using pytest.
It also includes some reusable utility pytest fixtures which makes creating new unit tests simpler and easier.
For a more in-depth description, please refer to the blog post Easy automated testing in Databricks