Projects
Each of these is a case study in what was at risk, and what changed once it was fixed.
Proving a Migration, Not Just Doing One
Ten source tables moving platforms. No way to prove any of them arrived intact.
- rows migrated on the pilot
- 678M
- automated checks per run
- 114
- average difference against source (2% tolerance)
- 0.01%
- model-tiered subagents behind one human approval gate
- 22
rows migrated on the pilot
automated checks per run
average difference against source (2% tolerance)
model-tiered subagents behind one human approval gate
- Databricks
- Lakeflow Declarative Pipelines
- Lakehouse Federation
- Snowflake
- Delta Lake (MERGE, Liquid Clustering)
- Medallion architecture
The Databricks Catalog Other Engineers Build Inside
Every engagement re-derived the same decisions. The knowledge lived in whoever was staffed.
- reusable skills, client-generic by construction
- 118
- categories, from platform core to AI and agents
- 13
reusable skills, client-generic by construction
categories, from platform core to AI and agents
- Databricks
- Python
- FastMCP
- Model Context Protocol
Proving Agent Skills Actually Work
A skill that passed once and a skill that works look identical from outside.
- meta-skills for authoring and evaluation
- ~24
- graduation bar over k independent runs
- pass^k ≥ 0.95
meta-skills for authoring and evaluation
graduation bar over k independent runs
- Python
- Model Context Protocol
- OAuth 2.1
- uv workspace
Architecture Enforced by Gates, Not Convention
Architecture documents describe intent. Codebases drift from it within months.
- packages split on a network-and-secrets axis
- 7
- AST gates failing CI on a boundary violation
- 4
packages split on a network-and-secrets axis
AST gates failing CI on a boundary violation
- Python
- PostgreSQL
- Databricks
- AST analysis
Statewide Transportation Data, Made Trustworthy
A feed lapsed. The dashboard still rendered, with a gap nobody could see.
- improvement in statewide reporting availability
- 30%
- faster onboarding for a new data source
- 60%
improvement in statewide reporting availability
faster onboarding for a new data source
- Databricks
- Delta Live Tables
- PySpark
- Azure OpenAI
The Metrics Behind a 50% EBITA Increase
Everyone had the data. Nobody had a reason to believe it.
- EBITA increase (~$50M) the pipelines contributed to
- 50%
- Databricks pipelines integrating IoT and SaaS sources
- 15+
EBITA increase (~$50M) the pipelines contributed to
Databricks pipelines integrating IoT and SaaS sources
- Databricks
- SQL
- Python
- Power BI
Costing a Migration Nobody Could Log Into
The scope doubled to a 2,000-task orchestration monolith. Access to the warehouse beneath it never arrived.
- orchestration tasks in the monolith being replaced
- 2,000
- hours estimated for the transformation component
- ~6,800
orchestration tasks in the monolith being replaced
hours estimated for the transformation component
- Azure Databricks
- Databricks Workflows
- Databricks Model Serving
- BigQuery
Hospital Billing, Modeled Once
The reports disagreed. The worse case was the one where they agreed and were still wrong.
- Epic Clarity cube views migrated (19 dimension, 7 fact)
- 26
- source keys reconciled, none unaccounted for
- 7.4M
Epic Clarity cube views migrated (19 dimension, 7 fact)
source keys reconciled, none unaccounted for
- Databricks
- Unity Catalog
- dbt
- Fivetran