Simplify Microsoft Fabric Deployments with “Deploy Microsoft Fabric items with fabric-cicd”, an Azure DevOps Extension

This post covers how you can simplify Microsoft Fabric deployments with “Deploy Microsoft Fabric items with fabric-cicd”. Which is an Azure DevOps extension that I recently published.

To manage expectations, this post shows how to start working with the extension and its associated task within the GUI-based classic release pipelines in Azure DevOps. Like in the below screenshot.

You can find examples of YAML pipelines in my ADO-deploy-fabric-items-task-examples repository. Along the way I share plenty of links in this post.

To get the most out of this extension, start with a basic understanding of fabric-cicd. My post on your first fabric-cicd deployment steps walks you through the fundamentals.

About the “Deploy Microsoft Fabric items with fabric-cicd” Azure DevOps Extension

The “Deploy Microsoft Fabric items with fabric-cicd” extension provides an easy-to-use pipeline task that automates the deployment of Microsoft Fabric workspace items directly from your source control repository. Whether you’re managing Notebooks, Data Pipelines, Semantic Models, or other Fabric items, this extension streamlines your deployment process with enterprise-grade authentication and flexible configuration options.

This extension leverages the fabric-cicd Python library to publish Microsoft Fabric items to your target workspace. It handles authentication via Azure Service Principal, installs the required dependencies, and executes the deployment based on your configuration.

Previously, using the fabric-cicd Python library in Azure DevOps pipelines required creating and maintaining custom Python scripts to orchestrate the deployment process. This extension eliminates that complexity by providing a ready-to-use pipeline task that handles dependency installation, authentication, and execution automatically.

You can start working with the task once you add the extension through the Azure DevOps Marketplace. When working with parameter and configuration files you need to add them to your own source control repository in Azure Repos.

I strongly recommend creating variables in at least one Azure Pipeline variable group to cater for your needs.

Getting started in a Classic release pipeline

When starting with a classic release pipeline, first add an artifact connected to your Git repository in Azure Repos. Like in the below example.

I recommend ensuring your source alias has a sensible name. Just in case you need to specify a file location in your release pipeline.

Next start adding tasks to your stage. You first need to add the ‘Use Python version‘ task to gracefully specify a version of Python between versions 3.9 and 3.12. So that the fabric-cicd library will work.

Afterwards, you can add the ‘Deploy Microsoft fabric items’ task.

‘Deploy Microsoft Fabric items’ task options

You will notice that all of the options contain useful tips when you click on the information icon above them. Because I strived to make the task as helpful as possible.

First you need to specify your authentication type with the drown down menu.

If you select “Service Connection”, you can select a valid service connection created for an Azure subscription or an Azure subscription itself.

I strongly recommend a service connection that is configured to use workload identity federation with openid connect. Richard Mintz wrote a post on how to create an Azure DevOps Service Connection that you can utilize to create such a service connection.

When working with service connections this way in Release pipelines always ensure that your Agent is configured to allow scripts to access the oath token.

If you decide you want to enter service principal details instead you can select “Service Principal” from the drop-down menu. When you select this option an additional three boxes appear so you can enter in the relevant service principal details.

Next, specify the environment required. One key point I want to highlight here is that when working with parameter and config files the environment names are case sensitive.

Next you can select if working with configuration-based deployments. Like I covered in a previous post. When selcted you can enter in a path relevant to the config file in your own GitHub repository. For example, ‘$(System.DefaultWorkingDirectory)/{Your artifact alias}/workspace/config.yml’.

For a more typical fabric-cicd deployment you can enter the target Workspace Id and the location of your repository directory. You must enter your repository directory location based on the ‘$(System.DefaultWorkingDirectory)/{Your artifact alias}/{location in repository}’

Pay special attention to the ‘Items in Scope’ option. If you leave this field empty, the task deploys all items stored in your repository directory. To deploy only specific items, enter their types as a comma-separated list without quotes, as shown in the example below.

Finally, you can also specify a version of fabric-cicd if you so wish. In the format of ‘0.1.{version number}’

Testing the Deploy Microsoft Fabric items with fabric-cicd extension

Once you are satisfied you can then run your pipeline. Like in the below example.

If you do encounter errors, then check the logs in your release. This can be a great location to go to confirm that your configuration-based deployments work as well. Apart from in the workspaces themselves of course.

Final words

I hope the “Deploy Microsoft Fabric items with fabric-cicd” Azure DevOps extension proves to be very beneficial to a lot of you. Because it took some effort to get it to this stage.

I must admit becoming a Visual Studio marketplace publisher has been an interesting experience. To see the end result has been very satisfying so far and I hope some of you feel the same when working with it.