This post covers important Microsoft Fabric CI/CD decisions to consider before or during a project. Because the Microsoft Fabric CI/CD approach can vary for different organizations. Getting the decisions right at an early stage reduces the complexity of your setup at a later stage.
Whilst sharing some technical information online about Fabric CI/CD recently, I received some interesting feedback. Someone mentioned that there is not enough information about the processes that are involved in Fabric CI/CD projects. That inspired me to write this post.
To manage expectation, this is a high-level overview of the more important Microsoft Fabric CI/CD decisions. As per the below image.
Along the way I share plenty of links to useful community resources and recent updates. Which means that the contents of this post can be new to those with prior Fabric CI/CD experience. If you want broader context about a certain topic, you are welcome to mention so in the comments section of this post.
Git integration or alternative
One key decision to make at the start of any Microsoft Fabric CI/CD journey is whether to implement Microsoft Fabric Git integration or an alternative for version control.
I personally think that Microsoft Fabric Git integration provides the most flexibility to work with a variety of CI/CD options. However, there are alternatives.
For example, you can store Power BI files in alternative locations like many did for years. You can also look at third-party offerings like Soterre For Power BI. Which handles source control for Power BI items differently.
Azure DevOps or GitHub
Following on from the version control topic is whether to work with Azure DevOps or GitHub.
Reason I only mention these two Application Lifecycle Management offerings is because these two are the only ones that currently support Microsoft Fabric Git integration and can orchestrate your deployments.
With Azure DevOps you can store your Git repositories in the Azure Repos service and orchestrate deployments with Azure Pipelines. Which you can do by pointing to your repositories as the source. Whereas in GitHub you store your Git repositories and manage orchestrate your deployments with GitHub Actions workflows.
Note that Microsoft Fabric Git integration handles authentication differently for the two providers. Azure Devops requires Microsoft Entra authentication. Whereas for GitHub you need to woks with Personal Access Tokens (PATs)
There are various factors to consider here. These include the platform your company currently uses and whether data sovereignty is required.
Previously, you could only specify a region for Azure DevOps. However, GitHub now supports the ability to deploy Git repositories to certain regions. I covered this in my previous post about recommended versions of GitHub.
You should also consider another option. Due to the fact that some folks in Microsoft are advocating migrating existing Git repositories stored in Azure Repos to GitHub. In order to take advantage of the Agentic AI capabilities in GitHub. You can watch the Azure DevOps and GitHub session from last years build to get a better understanding.
I must stress though that are still various ways you can work with AI in repositories stored in Azure DevOps. For example, the Microsoft Learn MCP server can help with a variety of tasks.
Feature or developer workspaces
Another common question that crops up when working with Microsoft Fabric Git integration relates to whether to implement feature workspaces as per the recommended development process or look to implement one workspace per developer.
This can depend on various factors within your environment. Feature workspaces are great to focus on individual changes. Even more so with the addition of the ability to only branch out only certain items that you select in the GUI.
There are various ways to create feature workspaces and branches. Including the GUI-based Branch-Out functionality provided by Microsoft and the branch out to new workspace scripts.
Workspaces for individual developers can be just as practical, especially in smaller teams. However, I would recommend ensuring they synchronize their branches on a per feature basis though to align with DevOps practices.
Manage deployments to other workspaces in or out of Microsoft Fabric
Another important Microsoft Fabric CI/CD decision to make is whether you are going to manage deployments to other workspaces in or out of Microsoft Fabric.
You can orchestrate deployments natively within Microsoft Fabric with Microsoft Fabric deployment pipelines. Microsoft formerly called these Power BI deployment pipelines.
Over time Microsoft has added additional functionality to the user experience (UX) for Microsoft Fabric Deployment Pipelines. For example, you can now add rules about notebook bindings. You can also achieve this with the Lakehouse Auto-Binding options in notebook settings.
You can also orchestrate outside of Microsoft Fabric in a variety of ways. For example, you can configure Microsoft Fabric Git integration to store your item metadata in a Git repository and then orchestrate deployments with either Azure Pipelines or GitHub Actions workflows.
You can do these types of deployments in many ways, Which I start covering in the next section.
I will finish off this section by adding that you can also consider a hybrid action. Orchestrating existing Microsoft Fabric deployment pipelines with either Azure Pipelines or GitHub Actions workflow. You can find an example of this in my ADO_OrchestrateFabricDP repository.
Merge Git branches or Git-based deployments
In reality, there’s many ways you can orchestrate Fabric CI/CD outside of Microsoft Fabric. However, one of the main Microsoft Fabric CI/CD decisions to consider when looking to perform this is whether to deploy updates by merging Git branches or with Git-based deployments.
You can update by merging Git branches by configuring Git integration in multiple workspaces connected to the same Git repository. For example, a dev workspace to a dev branch. From there, you can make a change, commit it and then merge it with a branch connected to another workspace to update that workspace.
You can also perform this with Git branches where you work with your Fabric items locally. For example, with Power BI Desktop. Like in the below diagram.
Hard to believe that I first posted about working with Git integration and multiple workspaces this way three years ago.
Git-based deployments also require you to store your Fabric items in a repository configured with Microsoft Fabric Git integration. However, deployment processes use these repositories solely as a source. This means that you do not need to configure Microsoft Fabric Git integration in your target workspaces.
Like in the below diagram which visualizes a deployment with the fabric-cicd python library.
One very common practice when looking to merge Git branches is to run scripts afterwards to perform rebinding’s and other essential tasks. For example, rebinding Power BI reports. There are plenty of examples of these scripts online. Including some in the Branch out to new workspace scripts by Microsoft.
In-reality, you need to do this for majority of Git-based deployments as well. Which I cover more about later in this post.
Core API’s or layer of abstraction
Once you have decided how you want to orchestrate your deployments outside of Azure DevOps you need to decide whether you want to work with the Microsoft Fabric API’s provided by Microsoft or a layer of abstraction.
Some people are comfortable working with many API calls in their orchestration scripts. In fact, a couple of folks have told me explicitly that they would love to see this covered more often.
However, a lot more people are keen on working with a layer of abstraction instead. Whether that be PowerShell or the fabric-cicd Python library.
Factors that can influence this decision include the skillset of the team responsible or a mandate that the solution should be as easy as possible. If the latter is preferred, then I recommend looking at a layer of abstraction instead.
Which Git-based deployment method to implement
In reality, there are a variety of Git-based deployment methods you can implement. Below is a subset of some examples:
- Microsoft Fabric API’s
- PowerShell
- Terraform
- Fabric-CLI
- fabric-cicd Python library
- Community offerings
One key point I want to highlight whilst you are making your important Microsoft Fabric CI/CD decisions is that the examples listed above are not mutually exclusive. You can work with a combination of them.
For example, you can deploy a Fabric workspace and populate it with items as part of your companies Infrastructure as Code (IaC) strategy. From there, orchestrate the deployments of the items and any updates to other fabric workspaces with the fabric-cicd Python library.
Key points to consider for Git-based deployment solutions
Below is some of the key points to consider when choosing which option(s) to work with:
- Existing company policies – For example, your company may require Terraform or PowerShell for all deployments.
- Teams’ ability to adopt the solution – You can come up with the most complete solution possible. However, you must be confident that the teams can adopt the solution or can be trained to do so.
- Can you apply the technology – Coming up with the solution is great, as long as it is feasible in the workplace. Because there can be some hidden gotchas. For example, working with security features like Azure Key Vault. I covered one workaround for this previously in a post about Managed DevOps Pools.
- Amount of flexibility required.
- Ease of adoption – For example, are the teams okay implementing fabric-cicd completely or would the fab deploy command be easier for them.
- Capabilities – Clearly you can do a lot with the core API’s if knowledge is present. Fabric CLI and Terraform both excel at deploying initial new items and fabric-cicd is very popular way to deploy those new items to other workspaces.
- More complex logic required in the pipeline – For example, do you need to manipulate the source files with PowerShell beforehand, or perform essential rebinding that fabric-cicd parameterization can handle.
- Possibility of adopting Azure DevOps extension- There are various extensions in-place. Including the Fabric automation tools Azure DevOps extension and the Deploy Microsoft Fabric items with fabric-cicd extension.
- Configuration-based deployments – Do you want the configuration for each workflow managed by the teams in their own repositories. If so, consider fabric-cicd configuration-based deployments.
In addition to the core architectural decisions, there are also decisions to be made how to develop, test and maintain the CI/CD processes.
AI-assisted Microsoft Fabric CI/CD development
Another important Microsoft Fabric CI/CD decision you must consider is whether to include AI assisted development. There are various factors to consider here, with the main one being if allowed in the environment.
If it is allowed, then I recommend you look into it accordingly. Because it can speed up development time and improve the feature set. Helping with tasks such as generating pipeline code and reviewing pull requests.
Some things to consider with AI-assisted development:
- Monitoring – In an enterprise environment you need to consider monitoring AI usage.
- Valid results – How will you validate the created items? For instance, testing that the created metadata is valid and there are no hallucinations.
- Costs – Depending on your AI solution on choice you will need to monitor the costs occurred.
- EU Artificial Intelligence Act – Will this act apply to you?
Implement automated tests
Another decision is whether or not to implement some form of automated tests and if so where? Based on my many years of DevOps experience I highly recommend that you do so to improve your Continuous Integration maturity level. For a variety of reasons.
One such reason is that identifying issues earlier on can save you a reputation and cost. As highlighted in the below diagram.
You can perform automated tests for a variety of items. Including Data Pipelines and Power BI reports.
For instance, Microsoft provides the Data factory – Testing Framework which allows you to automate the testing of Data Pipelines. Which can be adapted to automate testing Microsoft Fabric Data Pipelines with YAML Pipelines.
In addition, there are various options available to automate the testing of Power BI items. Including “PQL.Assert” developed by John Kerski. John wrote an article a while back that covers how you can build automated tests in Power BI with user-defined functions.
This leads me to another decision for some of the automated tests as well. Which is whether to perform the tests in or out of Microsoft Fabric?
For example, you can test your semantic model by directly inside your Azure Devops pipeline by running Tabular Editor CLI. Alternatively, you can get your Azure Devops pipeline to run a notebook that calls the semantic link library in Microsoft Fabric.
By calling the Tabular Editor CLI in your Azure Pipeline you keep the compute within the Azure Pipeline Agent. Whereas starting a notebook in Microsoft Fabric moves the compute to the underlying Microsoft Fabric capacity. Something to take into consideration.
Which Microsoft Fabric CI/CD resources to work with to get started
There are plenty of resources available to help you get started with your Microsoft Fabric CI/CD journey. Some say there is too much and it can be overwhelming.
I suggest you start by answering some of the questions in this post and then focusing on the relevant areas.
Microsoft provides a document that covers CI/CD workflow options as an alternative to some of the points covered in this post. Which was updated a while back.
Microsoft provides a variety of other documentation as well. One suggestion is to either find all the articles you need with your search engine of choice or work with the Microsoft Learn MCP server to get the answers you need.
In addition to the Microsoft articles there are various videos and articles available online that can help along the way.
Depending on the options you choose, various open-source solutions can help you get started. Some of these are listed in the Fabric Essentials Listings.
I share quite a few Microsoft Fabric related repositories to help others get started with their CI/CD journey. My only asks are that you give any repos that help a star and give my repositories a mention if utilized in your own material.
Epilogue
I hope this post about important Microsoft Fabric CI/CD decisions provides plenty of food for thought. This post highlights nine key decisions, although there can be more depending on your requirements.
You are welcome to put questions about any potential Microsoft Fabric CI/CD decisions in the comments section. So that either myself or another community member can assist you with them.
Do you have any preferences about any of the decisions? If so, you are also welcome to add those to the comments.