The Big Data Blog

We built the layers. We built the pipeline. Now it’s time to actually see the output —and make sure it’s governed, queryable, and ready...

What happens when your raw data doesn’t meet expectations? You filter. You clean. You level up. And sometimes, you drop 20,000 records...

Raw data is messy. But how  you ingest it sets the tone for everything downstream. In this week’s video, I walk you through creating the...

Data chaos doesn’t scale. Lack of governance kills velocity. That’s where Unity Catalog comes in. In this week’s cloud engineering...

In this week’s cloud engineering series video, I walk you through the process of setting up a Delta Live Pipeline  in Databricks to...

In this week’s cloud engineering series video, we’re finalizing our Gold Table in Databricks, the essential layer for reporting and...

In this week’s cloud engineering series video, we’re diving into error handling with an Error Logging Table in Databricks. Here’s why...

In this week’s cloud engineering series video, we take the Bronze table from last week and evolve it into a refined Silver table using...

In this week’s cloud engineering series video, we start our journey through the Medallion Architecture using Hive Metastore by building...

In this week’s cloud engineering series video, I show you how to quickly create a table from scratch using some lightweight, fake...

In this week’s cloud engineering series video, I walk you through the process of creating compute in Databricks—the essential engine that...

Kicking off my new cloud engineering series with a deep dive into Databricks metadata management. In this first video, I explain the key...

I’m thrilled to announce the launch of a new weekly series where I’ll be diving deep into the world of cloud data engineering—sharing...