Databricks auth: Service principal (OAuth M2M)¶

Recommended for production.

When to use it¶

Any production deployment, on any cloud (AWS, Azure, GCP).
You want an identity that does not depend on a person.

Trade-offs:

The OAuth secret must be stored, and rotated before it expires.
If you are on Azure and want the credential managed in Entra ID, use Entra ID service principal instead.
If you are on Azure AKS and want no secret at all, use Entra ID workload identity.

What you need¶

Workspace admin rights, to add a service principal.
Your workspace URL — see Workspace URL.

Step 1 — Add a Databricks managed service principal¶

In the steps below, choose Databricks managed at step 3.

In the Databricks workspace:

Click your user profile (top right) → Settings.
Go to Identity and access → Service principals → Add service principal.
Pick how the service principal is managed:
- Databricks managed — for Service principal (OAuth M2M).
- Microsoft Entra ID managed — for either Entra ID method. Paste the Entra application (client) ID into the Microsoft Entra application ID field.
Enter a Service principal name.
- Free text, and only a display label in Databricks.
- Use anything recognizable, for example trust3-collector.

Tick these workspace entitlements:

Entitlement	Why it is needed
Workspace access	Lets the service principal authenticate to the workspace.
Databricks SQL access	Needed to query the `system` catalog through a SQL warehouse for model usage traces.
Admin access	Covers all current and future workspace objects, so new clusters, jobs, serving endpoints and apps are discovered without a fresh grant each time. Also enables user email resolution on traces and ACL enrichment on assets.

Click Add service principal.

This registers the service principal in the account and assigns it to this workspace in one step.

Step 2 — Generate its OAuth secret¶

Open the service principal you just created.
Generate an OAuth secret.
Copy both values:
- Client ID — its application ID.
- Secret — shown only once.
Note the lifetime you chose. Authentication fails once the secret expires.

Step 3 — Fill in the collector form¶

In the collector's Datasource Configuration:

Field	What to enter
Databricks host	Your workspace URL
Authentication method	Service principal (OAuth M2M)
Service principal client ID	The client ID from step 2
Service principal client secret	The secret from step 2

Choosing the method reveals only these two credential fields and hides the rest.

Step 4 — Grant permissions¶

Grants are keyed on the service principal's application (client) ID:

The asset permissions in Assets permissions.
The system catalog grants below, if model usage collection is enabled.

Workspace entitlements are not Unity Catalog privileges. Even with Admin access, the identity needs these grants or the model usage and serving endpoint usage collectors return no rows.

Run them as a metastore admin or account admin.
Replace the principal with the identity the collector authenticates as.

SQL
-- Catalog / schema access
GRANT USE CATALOG ON CATALOG system TO `<collector-principal>`;
GRANT USE SCHEMA ON SCHEMA system.ai_gateway TO `<collector-principal>`;
GRANT USE SCHEMA ON SCHEMA system.serving TO `<collector-principal>`;

-- Table SELECT (model usage collectors)
GRANT SELECT ON TABLE system.ai_gateway.usage TO `<collector-principal>`;
GRANT SELECT ON TABLE system.serving.endpoint_usage TO `<collector-principal>`;
GRANT SELECT ON TABLE system.serving.served_entities TO `<collector-principal>`;

You also need a SQL warehouse for these queries:

Grant Can Use on it: SQL Warehouses → your warehouse → Permissions.
Copy its HTTP path from Connection details and enter it in the collector's SQL warehouse HTTP path field.