Skip to content

A coordinate on a dimension no data variable has breaks every query on the table #286

Open

Description

@alxmrs

A Dataset with a coordinate on a dimension that no data variable has registers without error, but then every query on it fails.

import numpy as np, xarray as xr, xarray_sql as xql

ds = xr.Dataset(
    {"t": (("x",), np.arange(3.0))},
    coords={"x": [0, 1, 2], "y": [10, 20]},  # no data variable has `y`
)
ctx = xql.Context()
ctx.from_dataset("ds", ds.chunk())
ctx.sql("SELECT x, t FROM ds ORDER BY x").to_pandas()

On main, unchunked: registration succeeds, and the query raises

DataFusion error: FFI error: Execution error: Failed to call stream factory: ValueError: Dimensions {'y'} do not exist. Expected one or more of FrozenMappingWarningOnValuesAccess({'x': 3})

Chunked (ds.chunk(), as above), from_dataset itself raises TypeError: unsupported operand type(s) for -: 'NoneType' and 'NoneType' in _block_len.

The schema and row count come from every dimension of the Dataset (ds.dims), while the blocks come from the data variables' dimensions. y has no place in t's rows.

Expected: the table for t has columns x and t and three rows, as ds[["t"]] (which drops y) gives. A fix could register each dimension group as ds[var_names], as the mixed-dimension path already does.

#275 sends such a Dataset to the lazy reader rather than to an in-memory table, so it behaves as on main, with a TODO naming this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions