fix(semantic-layer): derive dataset fqn from the table's Storage location (#761) by ondrejhlavacek · Pull Request #762 · keboola/cli

ondrejhlavacek · 2026-09-14T13:41:19Z

Closes #761.

Release Notes

semantic-layer add dataset and semantic-layer build now read the dataset fqn from the table's real warehouse location in Storage. Before, they wrote a hardcoded "KEBOOLA" database that exists in no project, so any consumer pasting the fqn into SQL failed (Kai, data apps, AI SQL generation).
New --fqn FQN on semantic-layer add dataset (and fqn in the REST body) stores a value verbatim.
storage table-detail returns two new keys: backend_path and sql_path.
semantic-layer validate --deep warns FQN_MISMATCH on datasets whose stored fqn differs from the table's location. Models built before this fix are flagged this way.

Plans for customer communication

None.

Impact analysis

Root cause. _derive_fqn returned "KEBOOLA"."<bucket-id>"."<table>". The Snowflake INFORMATION_SCHEMA query behind build --types-workspace also targeted "KEBOOLA".

Why not just swap the database name. I checked this in a read-only workspace on a GCP us-east4 project:

fqn	result
`"KEBOOLA"."out.c-azure-costs-copy"."2025-reservations-final"`	`Database 'KEBOOLA' does not exist`
`"KBC_USE4_5024"."out.c-azure-costs-copy"."2025-reservations-final"`	works
`"KBC_USE4_5024"."in.c-aws-costs-raw"."reports"` (linked bucket, own database)	`Schema ... does not exist`
`"KBC_USE4_337"."in.c-keboola-ex-aws-s3-…"."reports"` (linked bucket, source database and schema)	works

A linked bucket's tables live in the source project's database and schema. The Storage API returns exactly that in bucket.backendPath, so the fix uses it verbatim and never rebuilds the path from the tableId.

Changes

services/_table_detail.py: table_sql_path() quotes the location per backend. Snowflake gives "db"."schema"."table". BigQuery gives `dataset`.`table`, with the project prepended when databaseName is set; this matches bucket-detail. It returns None for an unsupported backend, a missing backendPath, or a quote character inside an identifier.
services/_semantic_layer_fqn.py (new): derive_dataset_fqn raises VALIDATION_ERROR when there is no usable location, pointing at semantic-layer add dataset --fqn (the same message serves build, which has no --fqn). append_fqn_mismatch_warnings powers the new validate --deep warning.
add_dataset fetches table detail once, which also serves --deep-fields. With --fqn alone it makes no Storage call.
build_information_schema_sql takes backend_path. The Snowflake query uses its database and schema. The BigQuery query uses backendPath[0] instead of rebuilding the dataset name from the bucket id.

Behavior changes

add dataset without --fqn now fails when the table does not exist, or when Storage reports no location for it. It used to store a broken fqn silently. The E2E tests that use synthetic tableIds now pass --fqn.
build aborts before any write when a table's location cannot be derived.
Existing models are not rewritten. gotchas.md documents the repair path: export, fix the fqn and metric sql, then import --overwrite.

Live verification (kbagent from this branch)

build --dry-run --types-workspace on a normal table and a linked alias table: both fqns match the working paths above. type_resolution_errors: [], and the linked table got its types from "KBC_USE4_337".INFORMATION_SCHEMA.
validate --deep on the two europe-west3 models from the issue: 41 FQN_MISMATCH warnings on each. The other 6 of the 47 datasets hit the existing DEEP_FETCH_FAILED.

Not covered. BigQuery was not verified live; no BigQuery project was available. A plain alias table in the project's own (non-linked) bucket was not verified live either: none of the available projects had one. It takes its own bucket's backendPath, the same as a regular table. build still emits Snowflake sql_dialect and column quoting on BigQuery, and its BigQuery INFORMATION_SCHEMA lookup omits the GCP project (unchanged from before). Both are tracked as a separate follow-up.

Reviews on this PR

Standards review (sub-agent against CONTRIBUTING.md and CLAUDE.md): no hard violations. Applied: documented the new table-detail keys in AGENT_CONTEXT, CLAUDE.md and keboola-expert.md; removed a duplicate quote check; trimmed comments that argued for the change.
Spec review (sub-agent against semantic-layer: dataset fqn hardcodes a "KEBOOLA" database that does not exist in the project (add dataset / build) #761): the build error advised --fqn, which build does not have. Fixed. A live SELECT 1 FROM <fqn> check in validate --deep was only suggested in the issue; this PR compares against the Storage location instead, which needs no workspace.
Gemini 3 Pro second opinion: no defects in the change. Its one note (BigQuery INFORMATION_SCHEMA without the GCP project) predates this PR and joins the BigQuery follow-up.

Change type

Bug fix (plus the additive --fqn option and new table-detail keys).

Justification

Every semantic model created through kbagent carried an fqn that resolves nowhere, which breaks every downstream SQL consumer of the semantic layer.

Deployment

Merge & automatic deploy.

Rollback plan

Revert of this PR.

Post release support plan

None.

…tion (#761) `semantic-layer add dataset` and `build` wrote every dataset fqn as "KEBOOLA"."<bucket-id>"."<table>". No project has a KEBOOLA database, so every consumer pasting the fqn into SQL failed. The INFORMATION_SCHEMA type lookup behind `build --types-workspace` queried the same nonexistent database. The location now comes from the owning bucket's Storage backendPath, exposed on `storage table-detail` as `backend_path` and `sql_path`. Swapping only the database would not be enough: a linked bucket's backendPath names the SOURCE project's database and schema, and only that path resolves (live-verified). - `add dataset` reads the fqn from table detail, so the table must exist; `--fqn` (CLI and REST body) stores a value verbatim instead. - A table whose location Storage does not report fails with VALIDATION_ERROR instead of getting a fabricated fqn. - `validate --deep` warns FQN_MISMATCH for stored fqns that differ, which flags models built before this fix.

keboola-pr-reviewer-bot

Verdict: auto_approve (risk 2/5) · profile keboola-mcp-server

Well-tested, well-documented bug fix deriving semantic-layer dataset fqn from real Storage location instead of a nonexistent "KEBOOLA" database.

- `build` no longer tells users to pass `--fqn`, a flag it does not have: the unresolvable-location error now points at `semantic-layer add dataset --fqn`. - Drop the second quote check in derive_dataset_fqn; `table_sql_path` already refuses identifiers it cannot quote, and the test now drives that path. - Document `backend_path` / `sql_path` on the table-detail entries of AGENT_CONTEXT and CLAUDE.md, and add the fqn gotcha to keboola-expert.md. - Trim docstrings and test comments that argued for the change.

Dismissing prior approval — a new commit was pushed and this review was for an earlier SHA. Run @keboola-pr-reviewer-bot review to get a fresh verdict.

keboola-pr-reviewer-bot · 2026-09-14T13:48:18Z

New commit on 5a94e0f — dismissed 1 stale bot approval. Comment @keboola-pr-reviewer-bot review when you want a fresh review.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

ondrejhlavacek · 2026-09-14T13:50:14Z

@keboola-pr-reviewer-bot review

keboola-pr-reviewer-bot

Verdict: needs_human (risk 4/5) · profile keboola-mcp-server

Head commit has failing checks (copilot-pull-request-reviewer); no required-check rules discovered on the base branch, falling back to block-on-any-failure.

soustruh

Approved. Root-cause fix: the dataset fqn is derived from the table's real Storage backendPath instead of a hardcoded KEBOOLA database. The change is additive and backward-compatible, and CI is green.

ondrejhlavacek requested review from keboola-pr-reviewer-bot and soustruh as code owners September 14, 2026 13:41

keboola-pr-reviewer-bot previously approved these changes Sep 14, 2026

View reviewed changes

keboola-pr-reviewer-bot added the agent-approved:keboola-mcp-server label Sep 14, 2026

ondrejhlavacek requested a lite review from Copilot September 14, 2026 13:48

Copilot started reviewing on behalf of ondrejhlavacek September 14, 2026 13:48

View session

Copilot AI reviewed Sep 14, 2026

keboola-pr-reviewer-bot reviewed Sep 14, 2026

View reviewed changes

keboola-pr-reviewer-bot added agent-needs-human agent-profile:keboola-mcp-server labels Sep 14, 2026

soustruh approved these changes Sep 21, 2026

View reviewed changes

ondrejhlavacek merged commit e805ee4 into main Sep 21, 2026
4 of 5 checks passed

ondrejhlavacek deleted the fix/761-semantic-layer-fqn branch September 21, 2026 18:55

soustruh mentioned this pull request Sep 24, 2026

chore(release): 0.95.0 #784

Merged

Navigation Menu

Repository navigation

fix(semantic-layer): derive dataset fqn from the table's Storage location (#761) - #762

fix(semantic-layer): derive dataset fqn from the table's Storage location (#761)#762
ondrejhlavacek merged 2 commits into
mainfrom
fix/761-semantic-layer-fqn

ondrejhlavacek commented Sep 14, 2026 •

edited

Loading

Uh oh!

keboola-pr-reviewer-bot left a comment

Uh oh!

keboola-pr-reviewer-bot commented Sep 14, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

ondrejhlavacek commented Sep 14, 2026

Uh oh!

keboola-pr-reviewer-bot left a comment

Uh oh!

soustruh left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

ondrejhlavacek commented Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Release Notes

Plans for customer communication

Impact analysis

Change type

Justification

Deployment

Rollback plan

Post release support plan

Uh oh!

keboola-pr-reviewer-bot left a comment

Choose a reason for hiding this comment

Uh oh!

keboola-pr-reviewer-bot commented Sep 14, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

ondrejhlavacek commented Sep 14, 2026

Uh oh!

keboola-pr-reviewer-bot left a comment

Choose a reason for hiding this comment

Uh oh!

soustruh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ondrejhlavacek commented Sep 14, 2026 •

edited

Loading