Recent Articles

All JSON Snowflake XML
Data Lineage,

BigQuery Data Lineage Guide – Features, API, & Limitations

In this post I dive deep into the BigQuery data lineage feature. For those of you who are not familiar with data lineage here is a quick primer.  Data lineage…
Uli Bethke February 2, 2025
Data Lineage,

Databricks Data Lineage – API, Tables, Unity Catalog

In this blog post, I’m taking the Databricks data lineage feature for a test ride. Let’s see what it can do and where you might need to rely on a…
Uli Bethke January 6, 2025
Data Lineage,

SQL Visualisation Guide – Query Diagrams, Lineage & ERD

Have you ever inherited the SQL codebase from someone else where it is in a “bit of a mess”? 😁 Where you find nested CTEs (Common Table Expressions), stacked like…
Uli Bethke November 3, 2024
Snowflake,

Iceberg Ahead! All you need to know about Snowflake’s Polaris Catalog

What is the Polaris Catalog? At the Snowflake Summit 2024 Snowflake’s CEO Sridhar Ramaswamy announced the Polaris Catalog during the main keynote speech. The announcement around the Polaris Catalog has…
Uli Bethke June 21, 2024
Data Pipeline,

Data Orchestration Deep Dive Snowflake Tasks. An Airflow replacement? 

What are Snowflake Tasks Snowflake introduced Tasks for scheduling and orchestrating data pipelines and workflows in late 2019. The first release of Tasks only offered the option to schedule SQL…
Uli Bethke May 3, 2024
SQL,

How to Parse XML Data in SQL Server

Are you still using fragile OPENXML and messy SQL to wrestle nested XML into relational tables on SQL Server? You know the drill: It starts as a simple script, but…
Uli Bethke January 25, 2024