Azure Key Vault and its Usage by using Data Factory

Introduction: All my articles are compiled into simple steps with detailed screen shots and elaborative description. By following these steps even, a novice programmer should be able to replicate scenario. Please go through the steps and provide your valuable feedback to make my next articles more clearer and intent.

Note: I took help of various internet sources while researching this item. Credit goes to internet and Microsoft communities.  

This is my 8th article on Azure.

Please follow below links for my previous Azure articles

Azure WebApps

Azure Logic App

Azure Event Grids

Azure SQL

Azure AD MSAL

Azure AD and User Management

Azure Key Vault With Logic App

Follow below mentioned 12 steps to Understand Azure Key vault creation and access it from Data Factory

 After completing this blog, you will be able to do

  • Set up Azure Key vault and secrets
  • Read value from Key vault using Data Factory
  • Microsoft Identity and its services

Sample Description: We are going to create a secret key using Azure Key vault and read its value by using Logic App

Step 1: Open Azure portal (https://portal.azure.com)

Step 2: Create New Azure Key Vault. Click on Create Resource > Search for Key Vault and Click on Create

What is Key vault: Azure Key Vault is a cloud service for securely storing and accessing secrets. A secret is anything that you want to tightly control access to, such as API keys, passwords, certificates, or cryptographic keys (From Microsoft).

Instead of using config files to store keys, we can use Azure key vaults. These are dynamic, secure and much controllable.

Step 3: We can store keys in key vault by using 3 ways – (a) Keys (b) secrets (c) certificates

In this article we are going to see how to create and share information by using secrets. We will also go through Access Policies.

Access Policies – Will allow key vault to provide and control its access to other applications like logic app, data factory, etc

Step 4: Now create a new Storage Account. After creating, Go to Access Keys section and grab the connection string.

Step 5: Create a new secret key with the connection string we copied from previous step.

Step 6: By using above secret key, we are going to access the Storage account.

Now create a Azure Data factory and connect to the storage account by using the secret key.

What is Azure Data Factory -Azure Data Factory is a managed cloud service that’s built for these complex hybrid extract-transform-load (ETL), extract-load-transform (ELT), and data integration projects.

Create Resource> Search for Data Factory >Create

Step 7: In next tab, Check “Configure Git Later” check box. We are not going to map any code in this article. This option can be used when data factory dealing with code

Step 8: After creating Data Factory, launch it and click on “Author & Monitor” button on home page- it will open Data Factory Interface. Click on Managebutton on left side menu and create a new Linked Service

Step 9: Before going further, go to Key vault and create access policy for Datafactory

Step 10: Now go back to data factory and create a new link service. First we need to create a linked service to get access to key vault.

In new Link service screen search for key vault and select

Step 11: Provide all the required information, select key vault populated and create a linked service. This linked service will provide access to the key vault.

Step 12: Create another new linked service. This linked service will be used to connect to the Azure Blob. Search for azure blob and continue.

IN next screen select Azure key vault instead of Connection string. That means now you are connecting to azure blob by using a secret key from vault.

IN AKV linked service select the linked service we created earlier, which gives access to the key vault.

In secret name section give the secret we created in key vault section.

And finally click on create . This data factory will be connected to Azure blob by using a secrete key.

In next articles, we will see how to use secret keys in dot net applications.

Happy Coding!!