Get started with the Custom Branch Out to New Workspace scripts for Microsoft Fabric

In this post I share how you can get started with the custom Branch Out to New Workspace scripts for Microsoft Fabric provided by Microsoft. Which you can find in the Fabric Toolbox GitHub repository provided by Microsoft.

This offering allows you to automate the “Branch out to new workspace” functionality with Azure DevOps. Which I covered in a previous post about version control.

Automating this process is ideal for various scenarios. Including when developers do not have permissions to create their own workspaces in Microsoft Fabric.

Below is a diagram that provides visualizes of how it works.

One key point I want to highlight is that the post activity notebook performs a variety of tasks. Including updating default and attached Lakehouses for notebooks and updating DirectLake model Lakehouse connections. I highly recommend going through the notebook to discover all of its capabilities.

I decided to do this post because I find myself recommending to others to look at this offering more often. In order for others to automate the process of branching out to new workspaces for others. Which can be very useful in environments where users do not have permissions to create their own workspaces.

Get started with the Custom Branch Out to New Workspace scripts for Microsoft Fabric

To get started I recommend creating either an Azure Key Vault to store a valid Fabric Token or create a parameter to add a valid token when you start the pipeline. Attempting to add a Fabric token as an Azure Pipelines variable will not work. Due to size limitations.

In addition, consider creating a valid Azure DevOps Personal Access Token and add it as a secret. As documented in the README file. You should create all the other non-sensitive variables recommended in the README file as well.

In addition, change the parameters in the “Branch_out_workspace.yml” file as required. For now, I recommend only authenticating with a Fabric token. Due to the fact that there are some issues when attempting to do this with a service principal.

Afterwards, you can continue with the rest of the instructions in the README file. However, before you look to start the pipeline, I do recommend amending the “Run_post_activity_py” file. I encountered issues passing through the parameter variables, so I had to add default values as below.

SOURCE_WS = args.SOURCE_WORKSPACE or ""
TARGET_WS = args.TARGET_WORKSPACE or ""
COPY_LH = args.COPY_LAKEHOUSE or ""
COPY_WH = args.COPY_WAREHOUSE or ""
CREATE_SC = args.CREATE_SHORTCUTS or ""
WS_ID = args.NOTEBOOK_WORKSPACE_ID or ""
NOTEBOOK_ID = args.NOTEBOOK_ID or ""
TENANT_ID = args.TENANT_ID or ""
CLIENT_ID = args.CLIENT_ID or ""
USERNAME = args.USER_NAME or ""
PASSWORD = args.PASSWORD or ""
CONNECTIONS_FROM_TO = args.CONNECTIONS_FROM_TO or ""
WH_VIEWS_ON_LH = args.WH_VIEWS_ON_LH or ""

Once done you can look to run the pipeline in Azure DevOps. Which should look like the below once completed.

One thing to be aware off is that the “Branch out to new workspace – Post Activity” notebook runs asynchronously in the current pipeline. So you must check its progress. You can do this in Microsoft Fabric by either going into the Monitor Hub or by looking at the run history of the notebook.

One thing I recommend is checking that the post activities specified in the notebook complete as expected. For example, check afterwards that the default lakehouses for notebooks now point to the lakehouses in the new workspace.

Service principal support for the Custom Branch Out to New Workspace scripts

In reality, there are many ways you can customize these scripts to suit your needs.

However, I must point out that even though you can branch out to a new workspace with a service principal the post activities will not work with the current scripts. Due to the fact that some of the semantic link functions in the notebook are not included amongst the list of functions that support service principals.

Final words

I hope this post encourages some of you to get started with the Custom Branch Out to New Workspace scripts for Microsoft Fabric. Because I want to raise awareness about this functionality. Due to the fact that I think this functionality will benefit a lot of you.

One thing I recommend doing is looking to customize it further. For example, looking to run the post activity notebook with the Fabric CLI instead so that it runs synchronously in the pipeline.