String-Case Data Quality Checks in Databricks Labs DQX

Status: Merged
Period: Merged upstream on 2026-07-29
Confidentiality: Public. Public merged contribution. I've kept the maintainer-review iteration visible.

Turned a text-quality requirement into a configurable DQX rule with explicit semantics, registration, documentation, and layered tests.

I added a reusable DQX check for uppercase, lowercase, title-case, and sentence-case validation. The contribution covers invalid configuration, non-string normalization, declarative registration, public documentation, unit and integration tests, and a review-driven follow-up that reduced test flakiness before the feature merged upstream.

Type: Open Source
Categories: Open Source
Data & Dashboards
Technologies: Python
PySpark
YAML
pytest

Context

Data-platform users and maintainers defining reusable quality rules in code or declarative configuration.

Supported modes and invalid configuration needed explicit semantics.
Coverage had to fit both unit and integration layers without claiming unreported performance or end-to-end testing.

Problem

DQX needed an unambiguous, reusable string-case rule that behaved consistently through programmatic and configuration-driven construction.

My contribution

I implemented four string-case modes, conversion and comparison behavior, invalid-mode rejection, declarative registration, documentation and agent-skill updates, and unit and integration coverage. I also refined tests in response to maintainer feedback.

Approach

Define the behavior contract first, implement normalization and validation through existing DQX patterns, register the rule for declarative construction, document it, and cover it across the repository's relevant test layers.

Outcomes

Merged upstream.
Added one configurable quality rule with four supported modes and both programmatic and declarative use.

Limitations and current status