Python SDK for Microsoft Fabric v0.1.2

It′s been a while that I revealed one of my side projects I have been working on: A Python SDK to work with the Microsoft Fabric APIs. Since then the Fabric APIs and as well the SDK (https://github.com/DaSenf1860/ms-fabric-sdk-core) has gone through a lot of new additions and changes.

With the addition of (nearly) all item specific APIs for CRUD operations, the Admin APIs as well as APIs for managing the Fabric capacity-resources in Azure, it was time to give an update on the recent developments.

Introduction

The Fabric APIs are very useful for a programmatic approach to get an overview of your items in Fabric as well as to change them. Using the APIs you can enforce standards in an automated way as well as keep an eye on your Microsoft Fabric estate. You could even use them to roll out something like Landing Zones or Project Templates.

The Python SDK is designed to make working with the APIs more convenient and more accessible as well as enhancing the functionalities of the APIs by building logic around them.

(Also check out my first article: https://www.dhirubhai.net/pulse/python-sdk-microsoft-fabric-andreas-rederer-geb-essbaumer--76edc)

For some examples and inspiration what you can do with it, check out the last section of this article.

What is in there at the moment?

With the addition of new Fabric APIs like the admin APIs, item specific APIs, One Lake Data Access Security, External Data Shares, .. as well as the Azure Resource Management APIs for Fabric Capacities, you can use the SDK to work with:

Core APIs to have full control over:

• Capacities (list capacities, assign & unassign them to & from workspaces, ...)
• Git (set up a workspace-gitrepo connection, trigger pulls and pushes, ...)
• Items (create, update, delete, list,...)
• Job Scheduler (run notebooks, pipelines, maintenance jobs,....)
• One Lake Shortcuts (create, delete, update,....)
• Workspaces (create, assign users, delete, update, ....)

Azure Resource Management APIs for Fabric capacities to handle Fabric capacity resources in Azure:

• Create capacities, suspend, resume, scale, add/remove administrators, list them,....

Admin APIs for ensuring governance:

• Domains (create, list, update them, assign workspaces to them, ...)
• Labels (assign/unassign sensitive labels at scale)
• Check Tenant Settings, User Assignments and External Data Shares,....

Item specific APIs to granularly control specific workspace items:

• List, create, update, delete warehouses, notebooks, semantic models, kql databases,...
• Lakehouse operations (Load table, list tables, run table maintenance)
• Creating your own spark environments (uploading libraries, configuring spark pools, ...)

We also added support of authentication through Fabric notebooks and we did a big refactoring to simplify the code and increase the performance significantly.

Of course we plan to also support new APIs which are not released yet

How to get started?

Installing the library is as easy as it gets:

pip install msfabricpysdkcore        

or you can check out the releases in the corresponding git repository

DaSenf1860/ms-fabric-sdk-core (github.com)

How to use it?

To work with the SDK you either work directly in a Fabric Notebook or initialize a Client first:

from msfabricpysdkcore import FabricClientCore

# Create a client

# Either work from a Fabric Notebook or login with the Azure CLI first and initiate the client directly
fc = FabricClientCore()

# Or use a service principal (note that not all APIs are supported with service principal)
# You can also use environment variables to set the service principal id and secret. The environment variables are:
# FABRIC_CLIENT_ID
# FABRIC_CLIENT_SECRET
# FABRIC_TENANT_ID

fc = FabricClientCore(tenant_id = "tenant_id",
                      client_id = "your_service_principal_id",
                      client_secret = "your_service_principal_secret")        

Now you can do all the things the REST APIs are doing and more.

E.g. creating, updating and listing workspaces, assigning capacities, build up a git connection, create shortcuts....

Here are some examples:

# Create a workspace
ws_created = fc.create_workspace(display_name="testworkspace",
                                 description="test workspace", 
                                 exists_ok=False)


# List workspaces
result = fc.list_workspaces()
for ws in result:
    print(ws)


# Get workspace role assignments
fc.get_workspace_role_assignments(workspace_id = ws.id)
# or
ws.get_role_assignments()        

Note that you can either provide the Client with all parameters or you can work on the object itself. E.g. a OneLake-Shortcut belongs to a Item which belongs to a Workspace. So it′s convenient that you can create an item on a workspace object and a shortcut on an item

ws = fc.create_workspace(display_name="testworkspace",
                         description="test workspace")

item = ws.create_item(display_name="lakehouse123", type="Lakehouse")


shortcut item.create_shortcut(path="path",
                              name="name",
                               target={"oneLake": {"itemId":                     
                                             "item_id_target",
                                            "path": "path_target",
                                            "workspaceId": 
                                              "workspace_id_target"}})        

What can I do with it? (some example scenarios)

Being able to work with the Fabric APIs in a programmatic way opens up a lot of possibilities for automatisms without interacting manually with the Fabric UI e.g:

• Design a "landing zone"-deployment by automating the following: Create a new capacity -> create a new domain -> create new workspaces -> assign the new capacity to these workspaces -> assign the workspaces to the domain -> assign users to the workspaces -> create a skeleton of workspace items in those workspaces

• Do a bulk re-assignment of workspaces from one capacity to another one. This also works for moving from a Power BI Premium capacity to a Fabric capacity (as long as it is in the same region and other limitations are considered).
• Do a bulk suspension of your sandbox capacities
• Do a bulk assignment of sensitivity labels for specific capacities, domains, workspaces, types of workspace items or any combination of those, ...

To see the code for those examples check out ms-fabric-sdk-core/usage_patterns.md at main · DaSenf1860/ms-fabric-sdk-core (github.com)

Conclusion

I hope this SDK helps some people to feel more confident to work with the Microsoft Fabric REST APIs and with Microsoft Fabric as their data analytics platform. My plan is to enhance and update the SDK whenever there are new API versions and new APIs released. I′m looking forward for some feedback. Feel free to reach out.

Notes:

This is not an official SDK from Microsoft. It is a community project and not supported by Microsoft.

Because this SDK uses the APIs in the background, all limitations and restrictions of the APIs apply to this SDK as well. This includes rate limits, permissions, etc.