Skip to content

Databricks auth: Personal access token (PAT)

When to use it

  • Getting started, a demo, or a short-lived evaluation.
  • Works on any cloud (AWS, Azure, GCP).

Avoid it for production:

  • The token acts as your user, with your permissions.
  • It stops working when that user leaves or loses access.
  • Tokens expire on the lifetime you set.

For production, use Service principal (OAuth M2M).

What you need

  • A Databricks user account with access to the workspace.
  • Your workspace URL — see Workspace URL.

Step 1 — Create the token

In the Databricks workspace:

  1. Open the user menu (avatar, top right) → Settings.
  2. Go to Developer → Access tokens → Manage.
    • Wording can vary by deployment; see the PAT documentation for your cloud.
  3. Choose Generate new token.
  4. Optionally set a comment and a lifetime (days until expiry).
  5. Under Scope, choose Other APIs (not BI Tools).

  6. In API scope(s), add all of these:

    API scope
    access-management
    apps
    clusters
    custom-llms
    genie
    jobs
    knowledge-assistants
    mlflow
    model-serving
    scim
    sql
    supervisor-agents
    unity-catalog
    vector-search

    mlflow and scim are required when trace collection is enabled (TRACE_COLLECTION_ENABLED=true) — for MLflow traces and for resolving user identities in trace metadata.

    Note

    If required scopes are not visible in the API scopes list, create the token using all APIs.

  7. Click Generate, then copy the token once while it is shown. It usually starts with dapi.

Step 2 — Fill in the collector form

In the collector's Datasource Configuration:

Field What to enter
Databricks host Your workspace URL
Authentication method Personal access token
Databricks token The token you copied

Selecting Personal access token hides the other methods' credential fields, so there is nothing else to fill in.

Step 3 — Grant permissions

The token inherits your user's permissions, so your user needs:

  • The asset permissions in Assets permissions.
  • The system catalog grants below, if model usage collection is enabled.

Workspace entitlements are not Unity Catalog privileges. Even with Admin access, the identity needs these grants or the model usage and serving endpoint usage collectors return no rows.

  • Run them as a metastore admin or account admin.
  • Replace the principal with the identity the collector authenticates as.
SQL
1
2
3
4
5
6
7
8
9
-- Catalog / schema access
GRANT USE CATALOG ON CATALOG system TO `<collector-principal>`;
GRANT USE SCHEMA ON SCHEMA system.ai_gateway TO `<collector-principal>`;
GRANT USE SCHEMA ON SCHEMA system.serving TO `<collector-principal>`;

-- Table SELECT (model usage collectors)
GRANT SELECT ON TABLE system.ai_gateway.usage TO `<collector-principal>`;
GRANT SELECT ON TABLE system.serving.endpoint_usage TO `<collector-principal>`;
GRANT SELECT ON TABLE system.serving.served_entities TO `<collector-principal>`;

You also need a SQL warehouse for these queries:

  • Grant Can Use on it: SQL Warehouses → your warehouse → Permissions.
  • Copy its HTTP path from Connection details and enter it in the collector's SQL warehouse HTTP path field.

For a PAT, the principal in those grants is the token owner's user name.

Read more