Integration Of Azure Databricks with Azure DevOps.

In this article I will demonstrate on how to integrate Databricks in Azure Devops.

Firstly, lets understand why we need to integrate both of these services.

In a normal software development project multiple developers work on their tasks. Its always good to have a source code repository which can be treated as a single source of truth. Also its easier to track the changes that changes when multiple developers are working. That's the whole point of having source control mechanism in place. Let's dig down on how to integrate Azure DevOps with Azure Databricks.

Steps :-

1) Click on the "Repos" option on the left side panel in Azure Databricks workspace and then click on "Add Repo" as shown below.

2. You will see a page as shown below.

3. Now assuming that you have created basic pre- requisites like Creating Organizations and Project in Azure DevOps, click on "Repos" and then create click on "New Repository" in Azure DevOps from the dropdown as highlighted.

4. Copy the URL as shown in the above screenshot, and paste it in the "Git Repository URL" field in Databricks which is shown in step 2. Once done, the Git Provider and Repository name will be populated automatically.

5.Scroll down in the same panel and select "Azure Devops Services (personal access token)"

6. Now next, we need to specify an access token which will be required to connect to Azure DevOps. Please find the steps for it.

a) Navigate to Azure DevOps portal and select "User Settings" from top right hand corner as highlighted. And from the drop down select "Personal Access Tokens".

b) Click on "New Token."

c) Enter a Token Name, and select "Full Access" under Scopes. as shown below.

d) Now you will get a secret token which is generated and you need to save it somewhere before you close the dialog box.

7. Now use this token and copy it in the "Token" section of the dialog box which was shown in Step 5. Now once you click in "Save", you will get a success notification as shown below. And then click on "Create Repo".

8. Now you would see the repository which was created in Azure DevOps has now been successfully integrated in Databricks.

9. From here click on "main" that will would take you to a page where you can create feature branch to make development. And then you can create a feature branch.

10) Now after you click on "Create" the repo will be now pointing to feature branch instead of main. And from here on you can start making development in your notebook and once you are done you can commit and push your changes. The changes would be now saved in the repository that we have now created.

I leave it to you to explore Step 10. !!!

So that concludes my article. Hope you had a understanding on how to connect Azure Databricks with Azure DevOps.

Thanks for reading. Keep learning !