Access Azure Key Vault secrets from MS Fabric Notebooks
Harshadeep Guggilla
Data Engineer | Microsoft Azure and Fabric | Cloudera On-Prem | ML, AI Enthusiast Trying to learn new things every day
Below are steps on how to fetch and use secrets from Azure Key Vault in a Microsoft Fabric environment. We will try to understand:
Why Use Azure Key Vault in Microsoft Fabric?
Azure Key Vault is a secure store for secrets, keys, and certificates. In a Fabric:
Method 1: Using a Service Principal (Client Secret)
Create an Azure Key Vault
Below is an Azure CLI example for creating a Key Vault. Adjust --name, --resource-group, and --location to your needs:
az keyvault create --name <unique-keyvault-name> \
--resource-group <resource-group-name> \
--location <azure-region>
Store a Secret in Key Vault
Use the az keyvault secret set command to add a secret. For example:
az keyvault secret set --vault-name <unique-keyvault-name> \
--name AZURE-AI-SERVICES-KEY \
--value "XXXXXXXXXXXXXXXXXXXXXXXXXXXX"
Create a Service Principal (SP) and Client Secret
Create the Service Principal
az ad sp create-for-rbac --name "<YourSPName>"
This command will output several important values:
Grant Key Vault Access to the SP In the Azure Portal (or via CLI), give the SP appropriate roles in Key Vault. Common roles:
Command approach below( or you might set an Access Control directly in the Key Vault settings if you’re using the older model)
az role assignment create \
--role "Key Vault Secrets User" \
--assignee <appId-of-SP> \
--scope /subscriptions/<subId>/resourceGroups/<rgName>/providers/Microsoft.KeyVault/vaults/<unique-keyvault-name>
Use the Service Principal in Fabric
In your Fabric notebook, install the required Python packages (if needed):
%pip install azure-identity azure-keyvault-secrets
Then authenticate using the ClientSecretCredential and retrieve your secret:
from azure.identity import ClientSecretCredential
from azure.keyvault.secrets import SecretClient
# Replace these values with your tenant/client details
tenant_id = ""
client_id = ""
client_secret = "" # place the secret here (or read from a secure source)
credential = ClientSecretCredential(tenant_id, client_id, client_secret)
vault_url = "https://fabric-poc.vault.azure.net/"
secret_client = SecretClient(vault_url=vault_url, credential=credential)
secret = secret_client.get_secret("fabric-secret-name")
print("Fetched secret:", secret.value)
Pros:
Cons:
Method 2: Using User Credentials (Interactive)
Because Fabric notebooks by default run under your user identity, you can leverage that token to fetch secrets from Key Vault. The snippet below uses the notebookutils.credentials.getSecret approach (the exact name may differ slightly depending on your Fabric environment):
key_vault_uri = 'https://fabric-poc.vault.azure.net/'
secret_name = 'fabric-secret-name'
secret = notebookutils.credentials.getSecret(key_vault_uri, secret_name)
print("Fetched password from key vault is", secret)
How it works:
Pros:
Cons:
Workspace Identity in Fabric: Current State & Limitations
In Azure, you could choose to run notebooks or pipelines as the system-assigned managed identity. However, Microsoft Fabric (in preview) does not yet expose that feature. Today:
Expected Future:
Final Thoughts and Best Practices
Let me know your thoughts on best approach :)