AI agents and Databricks: access control for tables and columns

Data is where many AI agent projects become serious. Answering a question such as "how many orders failed payment last week, per country" is a task an agent can do well, and it saves an analyst a small but constant stream of requests. At the same time, a data platform such as Databricks holds exactly the information that should not leave the company without a reason: customer records, payment details, employee data. An agent with a broad warehouse token can read all of it, and whatever it reads can end up in a prompt sent to a model provider.

This post describes how agent access to Databricks can be limited to specific tables and columns, why raw SQL is the wrong interface for agents, and how personal data can be masked before it reaches the model. It is part of our series on AI agents in the software development lifecycle.

Why direct warehouse access is a problem

The usual way to connect an agent to Databricks is a personal access token or a service principal with read rights on a catalog, together with a tool that executes SQL. This is quick to set up and works well in a demo. It has three weaknesses in production.

First, the scope is too wide. Warehouse permissions are usually granted per catalog or schema, for people who need to explore data. An agent that only has to answer questions about orders inherits access to every table in that schema, including those with personal data.

Second, raw SQL is a very expressive interface. An agent can join tables, write subqueries and select any column. Even with good intentions, it can combine data in ways nobody reviewed. If parts of the query come from user input or from text the agent read elsewhere, there is also the classic risk of injection.

Third, the result goes to the model. Every row the agent reads becomes part of its context, and with a hosted model this means it is sent to a third party. From a data protection perspective (for example under the GDPR), the question is therefore not only who may query a table, but which values may leave the network at all.

A narrower interface: structured, read-only queries

Vordix takes a different approach. Instead of accepting SQL, it offers four read-only operations:

• get_tables lists the tables the agent may use,
• describe_table returns the columns of an allowed table,
• query_table selects rows with filters, sorting and a row limit (capped by the gateway),
• aggregate_table computes grouped aggregates (count, sum, average, minimum, maximum).

The agent describes what it wants in structured parameters (table, columns, filters, grouping), and Vordix builds the SQL statement itself. Filter values are passed as bound parameters, never inserted into the query text. Joins, subqueries and raw SQL are not supported. Write operations do not exist for Databricks at all.

This is clearly less flexible than free SQL, and that is the intended trade-off. The interface covers the questions agents are usually asked (filtered lists, counts and sums per group) while making it impossible to query tables or columns outside the configuration.

Scoping by table and column

Tables are addressed with a full key of the form connection.catalog.schema.table. The connection part allows several named Databricks connections, for example one per workspace or environment. Vordix connects with OAuth machine-to-machine authentication through a service principal, so no personal token is involved.

Access is configured in two levels:

1. Tables. The administrator allows specific tables, at organisation level as a ceiling and then per project. A table that is not allowed does not appear in get_tables, and a query against it is denied.
2. Columns per table. Under each allowed table, the columns are an allowlist of their own. The allowed columns are checked against a stored scan of the warehouse schema. The value * means all columns of that table, which is practical for tables without sensitive content. For a customer table, a typical configuration would allow country, created_at and segment, but not email, name or iban.

Because columns are allowed per table, the same column name can be allowed in one table and blocked in another. The organisation-wide ceiling limits what a project can allow; a project cannot grant a table or column the organisation has not allowed.

Masking personal data in results

Column scoping removes the obvious fields, but personal data also appears in unexpected places: a free-text comment that contains an email address, a reference field with an IBAN, a log column with IP addresses. For these cases, Vordix applies data policies to the response before it is returned to the agent.

A data policy works in three layers:

• field-path rules for known fields (for example always mask customer.email),
• built-in detectors for common patterns: email addresses, phone numbers, IBANs, credit card numbers (validated with the Luhn check), IP addresses, and national ID numbers for Germany, Austria and Switzerland,
• organisation term lists for internal terms such as project code names.

For each match, the policy decides what happens: the value is masked, pseudonymised (replaced by a stable placeholder, so that the same customer still appears as the same entity), redacted, dropped, or the whole response is blocked. A preview tool shows what a policy does to a sample before it is enabled.

In addition, response fields can be filtered: only allowlisted fields of a response are returned, and if the filter cannot be applied, the call fails closed instead of returning unfiltered data.

Audit and rate limits

Every query is written to the audit log, including denied ones with the reason (for example "table not allowed" or "column not allowed"). Parameters are redacted before they are written, so the log itself does not become a second copy of sensitive filter values. Rate limits per key prevent an agent from turning into an unplanned bulk export. The post What an audit trail for AI agents should contain covers the log in more detail.

Trade-offs and limitations

The approach has clear limits, and it is better to know them before a project starts:

• No joins. Questions that need data from several tables cannot be answered in one call. The agent can query tables one after another, but for regular analyses a prepared table that already combines the data is the better option. This moves some work to the data team.
• No raw SQL, no writes. Advanced analytics, window functions or data changes are out of scope. The interface is meant for answering questions, not for data engineering.
• Detectors are pattern based. Email addresses or IBANs are recognised reliably; personal names in free text are not in the list of built-in detectors. Columns with names should therefore be excluded by the column allowlist or covered by field-path rules, not left to detection.
• Configuration effort. Choosing tables and columns is a data governance decision that needs someone who knows the data. The column wildcard * saves time, but it should only be used for tables where every current and future column is acceptable.

Conclusion

Giving AI agents access to Databricks is useful, but a warehouse token combined with free SQL gives them far more than the task needs. A narrower interface with structured, read-only queries, an allowlist of tables and of columns per table, and masking of personal data in results keeps the useful part (answers to routine data questions) and removes most of the exposure. The remaining limits, mainly the missing joins and pattern-based detection, are manageable if they are planned for.

More in this series: What is an MCP gateway? and What an audit trail for AI agents should contain. If you want to test the Databricks controls against your own warehouse, you can request a demo or read the Databricks integration page in the Vordix documentation.