YAML Pipeline examples for the “Deploy Microsoft Fabric items with fabric-cicd” Azure DevOps extension

This post shows YAML Pipeline examples for the “Deploy Microsoft Fabric items with fabric-cicd” Azure DevOps extension. Which is the Azure DevOps extension that I published last month and is free for everybody to work with within Azure DevOps.

You can find additional YAML pipelines examples in my ADO-deploy-fabric-items-task-examples repository. You can a guide on how to work with the extension in classic release pipelines in a previous post. For a similar GitHub alternative, I recommend the Deploy Microsoft Fabric items GitHub Action.

To get the most out of the “Deploy Microsoft Fabric items with fabric-cicd” extension, I recommend starting with a basic understanding of fabric-cicd. My post on your first fabric-cicd deployment steps walks you through the fundamentals.

Within this post I share plenty of links.

About the “Deploy Microsoft Fabric items with fabric-cicd” Azure DevOps Extension

The “Deploy Microsoft Fabric items with fabric-cicd” extension provides an easy-to-use pipeline task that automates the deployment of Microsoft Fabric workspace items directly from your source control repository.

Whether you’re managing Notebooks, Data Pipelines, Semantic Models, or other Fabric items, this extension streamlines your deployment process with enterprise-grade authentication and flexible configuration options. Making your CI/CD story easier when working with Azure DevOps.

This extension leverages the fabric-cicd Python library to publish Microsoft Fabric items to your target workspace. It handles authentication via Azure Service Principal, installs the required dependencies, and executes the deployment based on your configuration.

Previously, using the fabric-cicd Python library in Azure DevOps pipelines required creating and maintaining custom Python scripts to orchestrate the deployment process. This extension eliminates that complexity by providing a ready-to-use pipeline task that handles dependency installation, authentication, and execution automatically. Without the need of your own Python scripts.

You can start working with the task once you add the extension through the Azure DevOps Marketplace. When working with parameter and configuration files you need to add them to your own source control repository in Azure Repos.

I strongly recommend creating variables in at least one Azure Pipeline variable group to cater for your needs.

Getting started with YAML Pipeline

Getting Started with your YAML pipeline you should first add the relevant variable groups and Azure Pipelines agent pool. Like in the below example.

# This pipeline uses the variable groups below
# Feel free to create your own or use the below
variables:
- group: fabric-cicd-ns
- group: fabric-cicd-s

trigger: none
# In this pipeline I use a Microsoft-hosted agent
pool: 
  vmImage: 'windows-latest'

Once done you can create a new stage to define your deployment. One key point to note is that you first need to add the ‘Use Python version‘ task to gracefully specify a version of Python between versions 3.9 and 3.13. So that the fabric-cicd library will work within the custom task. Like the below example.

- stage: Test
  displayName: 'Deploy to Test'
  
  jobs:
  - job: 'DeployTest'
    displayName: 'Deploy To Test'

    steps:

      - task: UsePythonVersion@0
        displayName: 'Use Python 3.12'
        inputs:
          versionSpec: 3.12

Afterwards, you can add the ‘Deploy Microsoft fabric items’ task. You can do this by copying and pasting from one of my examples. Alternatively, you can add it through the assistant when editing the YAML pipeline in Azure DevOps.

I added helpful markdown text for each option. In addition, the view in the assistant can change depending on which option you choose. For example, when you change the authentication to service principal you can enter in the relevant service principal details.

I strived to make the process as easy as possible. Anyway, below are some YAML Pipeline examples to help you get started.

YAML Pipeline examples for the “Deploy Microsoft Fabric items with fabric-cicd” Azure DevOps extension

I will first start with a couple of examples based on manually entering the service principal details. Since that is the most common option.

Note that I recommend creating at the very least a “parameter.yml” file. In order to work with parameterization when you deploy items to new workspaces.

First example is how you can deploy all of the items that are stored in your Git folder in one go. In other words, deploying the metadata for all the items in your workspace at once.

# Note that when working with parameter and/or config files environment names are case sensitive
- task: DeployMicrosoftFabricItems@0
   inputs:
      azureClientId: $(AZURE_CLIENT_ID)
      azureClientSecret: $(AZURE_CLIENT_SECRET)
      azureTenantId: $(AZURE_TENANT_ID)
      workspaceId: "$(workspaceId)"
      repositoryDirectory: '$(Build.SourcesDirectory)/workspace'
      environmentName: 'test'

If you need to deploy certain items instead you can specify the items in scope.

- task: DeployMicrosoftFabricItems@0
   inputs:
       azureClientId: $(AZURE_CLIENT_ID)
       azureClientSecret: $(AZURE_CLIENT_SECRET)
       azureTenantId: $(AZURE_TENANT_ID)
       environmentName: 'test'
       workspaceId: "$(workspaceId)"
       repositoryDirectory: '$(Build.SourcesDirectory)/workspace'
       itemsInScope: $(ItemsInScope)

One way of working with fabric-cicd that is becoming more popular is working with configuration-based deployments. Which is where you manage your deployments with a central configuration file. You can specify the configuration file in the task as below.

- task: DeployMicrosoftFabricItems@0
   inputs:
      azureClientId: $(AZURE_CLIENT_ID)
      azureClientSecret: $(AZURE_CLIENT_SECRET)
      azureTenantId: $(AZURE_TENANT_ID)
      includeConfig: true
      configFilePath: './workspace/config.yml'
      environmentName: 'test'

Take note that you need the includeConfig flag set to work with configuration-based deployments.

This extension supports multiple types of authentication methods. Including Azure Pipeline Service Connections. In fact, it supports service connections configured with Workload Identity Federation.

Richard Mintz wrote a post on how to create an Azure DevOps Service Connection. One key point to add to this is that the name of the created workload identity needs to be granted access to the destination workspaces.

Below is an example of how you can deploy all items when working with a service connection configured with Workload Identity Federation.

- task: DeployMicrosoftFabricItems@0
   env:
      SYSTEM_ACCESSTOKEN: $(System.AccessToken)
   inputs:
      azureSubscription: $(azureSubscription)
      workspaceId: $(workspaceId)
      repositoryDirectory: '$(Build.SourcesDirectory)/workspace'
      environmentName: 'test'

I find specifying “System.AccessToken” within the task itself the most effective way to work with Workload identity Federation. Feel free to test other methods.

You can work with the other parameters just like when manually adding the service principal details. For example, you can specify the configuration file as below.

- task: DeployMicrosoftFabricItems@0
   env:
       SYSTEM_ACCESSTOKEN: $(System.AccessToken)
   inputs:
      azureSubscription: $(azureSubscription)
      includeConfig: true
      configFilePath: './workspace/config.yml'
      environmentName: 'test'

As you can see, the only option that changes is the authentication method.

Tips when working with the “Deploy Microsoft Fabric items with fabric-cicd” Azure DevOps extension

Once your task has been added you can run your pipeline. Below are some tips to help you improve your experience:

• If in doubt, work with the markdown tips I added in the assistant.
• This extension works with both Linux and Windows Azure Pipeline Agents. Choose whichever one works for your pipeline.
• Work with Workload Identity Federation service connections where possible. To avoid recycling secrets.
• Get into the practice of never exposing sensitive details in your pipeline.
• Depending on your authentication method create up to two pipelines. One for sensitive(s) and another for non-sensitive(ns) variables.
• If in doubt, refer to my ADO-deploy-fabric-items-task-examples repository.

Final words

I hope these YAML Pipeline examples for the “Deploy Microsoft Fabric items with fabric-cicd” Azure DevOps extension helps speeds up adoption.

Feel free to rate and review the Azure DevOps extension once you start working with it. Because it took a reasonable amount of effort to develop this for the community.