Skip to content

Engine introspection is not working with SQLAlchemy==1.3.24 #171

Closed

Description

@bkyryliuk

Repro:

from sqlalchemy import create_engine
from sqlalchemy.engine.reflection import Inspector
engine = create_engine(f'databricks://token:{TOCKEN}@dropbox-prod.cloud.databricks.com:443/default?catalog=hive_metastore', connect_args={"http_path":"/sql/1.0/warehouses/{ID}"})
inspector = Inspector.from_engine(engine)
columns = inspector.get_columns(f"{catalog}.{schema}.{table}")

Error mgs:

AttributeError: 'DatabricksDialect' object has no attribute 'get_driver_connection'

Activity

  1. susodapop commented on Jul 10, 2023

    @susodapop
    Contributor

    Thanks for this report. I've opened a pull request that should fix this for you. You can even test it out using a dev build I just published to pypi: pip install databricks-sql-connector==2.7.1.dev1

    You need to modify your repro to use the connection string that databricks-sql-python expects (this is addressed in the PR description), but otherwise it runs like it should.

    from sqlalchemy import create_engine
    from sqlalchemy.engine.reflection import Inspector
    
    
    host="****.cloud.databricks.com"
    http_path="/sql/1.0/warehouses/****"
    access_token="dapi****"
    catalog="samples"
    schema="nyctaxi"
    
    engine = create_engine(f'databricks://token:{access_token}@{host}', connect_args={"http_path": http_path, "catalog": catalog, "schema": schema})
    inspector = Inspector.from_engine(engine)
    columns = inspector.get_columns("trips", schema="nyctaxi")

    Note that you should include the schema in your calls to Inspector.get_columns(). I even added a smoke test to our e2e suite so we can tell if this breaks in the future.

  2. self-assigned this
    on Jul 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions