Requirement
Let’s say we have an Azure Key Vault where we used to store the secrets. We want to access these secrets in Azure Databricks.
Prerequisite
Before going further, you should have:
Solution
Step 1: Create a secret in Azure Key Vault
Go to Key Vault and create a dummyKey secret with any value.
Step 2: Create a Notebook in Azure Databricks
Launch Databricks workspace and create a notebook.
Here, I have created a notebook named AccessKeyVault with the Scala language.
Step 3: Access Key Vault from Azure Databricks
For accessing Key Vault in Databricks, we have to use the Secret Scope which we had created in the last post. If you have not created, you can see Create Secret Scope in Azure Databricks post.
In our case, we had created a scope named databricks-secret-scope. Now, we want to access the secret of the key named dummyKey which we have created in step -1.
Databricks provide a method called get which takes 2 parameters – Secret Scope and Key.
val source = dbutils.secrets.get(scope = "databricks-secret-scope", key = "dummyKey")
It will give return a string like source: String = [REDACTED] which means it has read the secret successfully.
Wrapping Up
In this post, we have accessed the secret stored in the Azure Key Vault. This is very important as we use multiple cloud services in Azure and required to access securely.