Skip to content

fix(e6): map empty identifiers to empty_alias on the native executor - #323

Open
tkaunlaky-e6 wants to merge 2 commits into
mainfrom
fix/e6-native-empty-alias
Open

tkaunlaky-e6 wants to merge 2 commits into
mainfrom
fix/e6-native-empty-alias

Conversation

@tkaunlaky-e6

@tkaunlaky-e6 tkaunlaky-e6 commented Oct 5, 2026 •

Copy link
Copy Markdown

Problem

Databricks treats `` (an empty name) as a normal column name. Checked on Spark 4.2, ANSI mode on and off:

  • count(*) AS `` produces a column named '';
  • that column can be referenced by `` everywhere: outer SELECT, qualified `s.```, through a CTE, `ORDER BY`, `GROUP BY`.

e6 cannot parse an empty identifier, so the transpiled AS "" failed with Encountered "\"". This is the "double-quoted string" variant in the TomTom native coverage sheet: 2 statements, one of which ran OK on Databricks.

Fix

With E6_EXECUTOR_TYPE=native, identifier_sql writes every empty identifier as empty_alias. Because the alias and every reference to it get the same name, they stay linked, e.g.

SELECT `` FROM (SELECT count(*) AS `` FROM t) s ORDER BY `` → SELECT empty_alias FROM (SELECT COUNT(*) AS empty_alias FROM t) AS s ORDER BY empty_alias

The java path is unchanged.

Behaviour vs Databricks

Databricks (Spark 4.2) e6 with this fix
Values, all shapes below ✓ same as Databricks
Column name '' empty_alias
Two empty-named columns, then referenced by name AMBIGUOUS_REFERENCE error Column 'empty_alias' is ambiguous error

Shapes checked on both, with the same results: single and multiple empty aliases, empty alias on a plain column, subquery and CTE with SELECT *, outer SELECT ```, qualified s., CTE column referenced by empty name, `ORDER BY , GROUP BY ```, GROUP BYwithORDER BYordinals,UNION ALL`, and window functions.

Testing

  • make check passes: ruff, ruff-format, mypy, and the full suite with both tokenizers (1013 tests each).
  • test_empty_alias_native_executor covers the alias, a reference through a subquery plus ORDER BY, a normal alias (unchanged), and java (unchanged). 2 of its checks fail without the fix.
  • The sheet's sample now gets past the parse error on a local e6 engine.

e6 rejects an empty identifier, so a Databricks `count(*) AS `` ` came out as
`COUNT(*) AS ""` and failed to parse (Encountered "\""). With
E6_EXECUTOR_TYPE=native, an empty alias is now dropped and the engine names
the column (e.g. EXPR$1); the values are unchanged. The java path is unchanged.
Dropping an empty alias broke queries that refer to that column by its empty
name, e.g. SELECT `` FROM (SELECT count(*) AS `` ...): Databricks resolves ``
as a normal column name (in SELECT, qualified s.``, CTEs, ORDER BY, GROUP BY),
but e6 cannot parse "". On the native executor every empty identifier is now
emitted as empty_alias, so the alias and its references resolve to the same
column. Values match Databricks; only the column name differs ('' there,
empty_alias here). Two empty-named columns referenced by name are ambiguous
in both engines.
@tkaunlaky-e6 tkaunlaky-e6 changed the title fix(e6): drop an empty alias on the native executor fix(e6): map empty identifiers to empty_alias on the native executor Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant