Requirement
The security is the main concern when you are working with cloud services. Azure provides a service to store all the secrets and retrieve them when and where needed. This feature is called Key Vault. The databricks provides a feature to store all the secrets called Secret Scope.
There are two types of secret scope available in Azure Databricks:
Azure Key Vault-backed: You can create a secret scope backed by Azure Key Vault and leverage all the secrets created in the Key Vault using this Secret Scope.
Databricks-backed: This is a store in the encrypted database owned and managed by Azure Databricks.
In this post, we are going to create a secret scope in Azure Databricks.
Solution
Here is the prerequisite:
- Azure Subscription (If don’t have then check )
- Azure Key Vault
- Azure Databricks
Step 1: Login to Azure Portal
Go to portal.azure.com and login with your credential.
Step 2: Get Databricks Instance
Go to Databricks cluster and copy the URL.
In browser URL: <databricks-URL>#secrets/createScope, below page will be open.
Step 3: Get DNS & Resource ID from Key Vault
Go to Key Vault portal. Find the properties from the left bar. Get the DNS and Resource ID:
Step 4: Create Secret Scope
Choose All Users under Manage Principal if all team members want to use the same secret scope.
Once you click on the Create, the above pop up will come and will show that the secret scope has been created.
Wrapping
In this post, we have learned how to create secret scope in Azure Databricks. We will see how can we leverage this secret scope in our next post.