Azure Databricks: Installing, Configuring and Using the Azure Databricks CLI with multiples workspaces

Some months ago I started to work with Azure Databricks and I was learning many things about Databricks from scratch. I think that working with Databricks is not easy because there is not a lot of documentation about it. It is a young platform then what better way than to share what little we know.

Databricks CLI

The Databricks command-line interface (CLI) provides an easy-to-use interface to the Databricks platform. The best way to manage Databricks is using the CLI interface. You can create new clusters, jobs, execute tasks, import and export workspaces. Before installing the Databricks CLI library you need to already have requirements done.

Requirements

For install Databricks client you need to comply with a least one follow points:

• Python 3–3.6 and above
• Python 2–2.7.9 and above

You can check your python version using this command

python  - -version

Install Databricks CLI library

To install the library you only need to execute this command in your cmd

pip install databricks-cli

After you have successfully installed the CLI you can check Databricks version using

databricks -v

Databricks CLI allows you to display commands to execute with CLI help. You can see all available options

databricks -h

CLI output to show you commands in each command group.

Generate Databricks Token

First you need to create an access token for every workspace. I’ll teach you how to create one access token and you might follow this steps:

1. Go to Azure Databricks main page and you should see something like an attachment screenshot. After that you have to select the User Settings option.

2. In User Setting Option you should generate a new token.

3. This step is very important because you can fill forms using an intuitive comment and lifetime what you want to live that token.

4. When you select Generate, you should copy and paste the token value into secure places.

5. Lastly, your new token should be in the list of created tokens.

Connection profiles

The Databricks CLI configuration supports multiple connection profiles. The same installation of Databricks CLI can be used to make API calls on multiple Databricks workspaces.

databricks configure [--profile <profile>]

Adding a connection profile for every workspace. Remember to generate a token by workspace:

databricks configure --token --profile demotest_dev

databricks configure --token --profile demotest_pro

Databricks supports many commands group that you can use with the connection profile:

Commands group

clusters        Utility to interact with Databricks clusters.
configure       Configures host and authentication info for the CLI.
fs              Utility to interact with DBFS.
groups          Utility to interact with Databricks groups.
instance-pools  Utility to interact with instance pools.
jobs            Utility to interact with jobs.
libraries       Utility to interact with libraries.
runs            Utility to interact with the jobs runs.
secrets         Utility to interact with Databricks secret API.
stack           [Beta] Deploy and download resource stacks.

Execute commands group with specific profile

databricks <commands group> --profile <profileName>

PS if you have any questions, or would like something clarified, you can find me on Instagram, Twitter and LinkedIn. Also If you find any error let me know.