Testing fab deploy with the new Fabric automation tools Azure DevOps extension

This post covers testing fab deploy with the new Fabric automation tools Azure DevOps extension. Plus, I show how you can authenticate with the new Azure Devops extension in various different ways.

Since it has just been released there are a few quirks with the new extension. In fact, some there are already some GitHub issues raised for it. My intention is to show you how you can currently work with it to deploy with the fab deploy command.

Plus, I show you some insights into the extension itself since the code is shared openly in a GitHub repository. Leaning on my own experience of being an Azure DevOps extension creator. You can view some YAML Pipeline examples for my Azure DevOps extension in a recent post.

Please note that everything in this post is accurate at the time of publishing.

About the new Fabric automation tools Azure DevOps extension, fab deploy and fabric-cicd

Fabric automation tools is a new Azure DevOps extension that has been released by Microsoft. Which provisions the Fabric CLI (fab) into the pipeline agent. You can consider it a wrapper for one or more fabric CLI commands and includes authentication at the same time.

You can install the extension through the Azure DevOps marketplace.

Fab deploy is a new Fabric CLI command that allows you to use the fabric-cicd Python library directly within Fabric CLI in order to deploy Microsoft Fabric items into Microsoft Fabric workspaces. Without the need to install the additional fabric-cicd library.

Fabric-cicd is a Python library that allows you to perform CI/CD of various Microsoft Fabric items into Microsoft Fabric workspaces. With some customization it can resolve issues like rebinding items to newly deployed items along the way.

As you can see from the descriptions above, using fab deploy with the new Fabric Automation Tools Azure DevOps extension is a bit like a Russian nesting doll; you’re using an extension that wraps a command, which itself wraps a popular Python library, which in turn wraps around API’s.

Testing fab deploy with the new Fabric automation tools Azure DevOps extension

I covered using fab deploy in Azure DevOps for Microsoft Fabric deployments in a previous post. So i decided to be efficient and use the same fab deploy command in the same Git repository.

In the overview for the extension it states that authentication uses workload identity federation. So that the federated token is generated at pipeline runtime. However, I was keen to check if there are other ways to authenticate.

So, I navigated to the ms-fabric-azure-devops-extensions Git repository. Which contains the source code for the extension. From there, I first navigated to the ‘task.json‘ file. Which shows the correct task name as “FabricCLI”.

After looking through the task file I navigated to the “index.js” file. This is where things started to get interesting. Because this file clearly shows that multiple authentication types are supported. As you can see below.

const FABRIC_AUTH_ENV_VARS = [
    'FAB_TOKEN',
    'FAB_TOKEN_ONELAKE',
    'FAB_TOKEN_AZURE',
    'FAB_TENANT_ID',
    'FAB_SPN_CLIENT_ID',
    'FAB_SPN_CLIENT_SECRET',
    'FAB_SPN_CERT_PATH',
    'FAB_SPN_CERT_PASSWORD',
    'FAB_SPN_FEDERATED_TOKEN',
    'FAB_MANAGED_IDENTITY',
];

As looks like you can specify the App Id, Tenant Id and secret value for a service principal instead of working with workload identity federation. In addition to other authentication methods.

Once I had finished looking through some of the other files in the Git repository, I decided to make a start on my YAML Pipeline.

First, I opted to enter in the service principal credentials instead of working with workload identity federation. To confirm that my findings were correct.

I then had to rotate until I found a script language that worked. I then had to resolve an error that appeared in the log relating to the encrypted cache. Luckily, I had encountered that issue in the past. As shown in a previous post about automating FUAM deployments. My working version of the YAML pipeline is as below.

trigger: none

pool:
  vmImage: ubuntu-latest

variables:
- group: fabric-cicd-s
- name: TargetEnvironment
  value: Test

stages:
- stage: deploy
  displayName: Deploy to workspace
  jobs:
  - job: fabdeploy
    displayName: Use fab deploy
    steps:

    - task: FabricCLI@0
      displayName: 'Use fab deploy'
      env:
        FAB_SPN_CLIENT_ID: $(AZURE_CLIENT_ID)
        FAB_TENANT_ID: $(AZURE_TENANT_ID)
        FAB_SPN_CLIENT_SECRET: $(AZURE_CLIENT_SECRET)
      inputs:
        scriptLanguage: 'bash'
        scriptType: 'inlineScript'
        inlineScript: |
          fab config set encryption_fallback_enabled true
          fab deploy --config "$(Build.SourcesDirectory)/workspace/config.yml" --target_env "$(TargetEnvironment)" -f
        FabricCLIVersion: 'v1.5.0'

As you can see in the above example you can add multiple fab commands in the same task. Once the pipeline had completed, I checked the YAML Pipeline job details. Scrolling down in the task details to check that all the deployments had taken place.

Final thoughts

As you can see from this post, you can get the new Fabric automation tools Azure DevOps extension to work. However, it is new so expect some issues.

I was a bit surprised that you can authenticate in other ways within the code. Hopefully, the maintainers will update the documentation to highlight the other methods as well. Instead of opting to remove them.

I felt a bit guilty whilst going through the source code in the GitHub repository. Because the main reason I decided not to share the source code for my extension yet is because I did not want to see multiple extensions appear that are similar to my own. Due to the effort involved in developing my extension.

However, it does mean that I was able to share some interesting findings with you all.

Right now, you can authenticate once and issue multiple fab commands in the same task. However, one thing I want to see in the future are tasks that no longer need the full fab {blah,blah} command. Because I think a lot of people would appreciate separate tasks for the various fab commands.

Especially if there is a specific task that allows you to authenticate once per stage instead of for each task. Making any additional tasks specified in that stage nice and compact. Time will tell.