Best Practices for Using Data Fabric Workspaces + Git
Effective management of data pipelines is key to ensuring continuous development, testing, and deployment processes. Integrating Git with Microsoft Fabric workspaces offers a…
Ahead of the AI
Articles
Databricks Lakehouse Federation: Exploring Practical Examples
Following up on the previous article, “Databricks Lakehouse Federation: An Alternative to Zero ETL?”, let’s explore some practical examples of combining data from different sources.
Databricks Lakehouse Federation: An Alternative to Zero ETL?
Released in December 2023, the Lakehouse Federation feature is currently in Public Preview, but it has enormous potential. In today’s article, I’ll explore some interesting perspectives, such as the ability to…
🚀 Continuous Integration and Continuous Delivery (CI/CD) in Azure Data Factory! 🔁✨
CI is the practice of automatically testing every code change, while CD deploys those changes to staging or production systems. In Azure Data Factory, CI/CD involves moving pipelines between…
Databricks: Best Practices for Managing Bronze, Silver and Gold (Medallion Architecture)
Many of the clients I work with implement a Medallion architecture to logically organize their data in a Lakehouse. As data flows, it moves through several stages or layers. The best-known design…
🔵✨ Azure Data Factory and Azure Databricks Best Practices ✨🔵
Hello everyone, this week we’ll go over some best practices for a data ingestion pipeline:
Hands-on with Microsoft Fabric: A Beginner's Guide (Part 1)
Traditional large-scale data analytics solutions rely on data warehouses and SQL queries to store and retrieve data. However, the rise of big data, characterized by large #volumes …
🚀 Key Differences Between Azure Synapse and Databricks
Hi everyone, in this article I’ll talk a bit about these data platforms.
Databricks Tip: How to Track Where Each Record Came From, File by File - input_file_name()
Wouldn’t it be a dream to have a column in your Bronze table (or Landing, whatever you prefer to call it) telling you which file each record came from?
Azure Synapse Analytics vs Microsoft Fabric: How About Another Migration Project?
In May 2023, Microsoft announced Microsoft Fabric. This new solution extends the integration promise made in Azure Synapse Analytics to cover every analytics workload, and it can be used…
How to Optimize SQL Queries: A Pocket Guide to Data "Tuning"
Time goes by, new data ingestion techniques, languages and tools come along, but one thing is a universal truth. SQL never goes out of style!
How to Use Dynamic SQL in BigQuery with EXECUTE IMMEDIATE
This week (June 2020) a GCP update came out that lets you use the EXECUTE IMMEDIATE command in BigQuery. For more information, see the documentation.