Skip to content

Enable TableProviderFactories to receive additional options when creating an external table - #4126

Merged
alamb merged 1 commit into
apache:masterfrom
timvw:feat/create-external-table-options
Nov 8, 2022
Merged

alamb merged 1 commit into
apache:masterfrom
timvw:feat/create-external-table-options

Conversation

@timvw

@timvw timvw commented Nov 7, 2022

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #4125.

Are there any user-facing changes?

The user can now provide additional options (depending on the tableprovider(factory)) when creating an external table

@github-actions github-actions Bot added logical-expr Logical plan and expressions sql SQL Planner core Core DataFusion crate labels Nov 7, 2022
@timvw timvw changed the title initial work to accept options in create external table statement Enable TableProviderFactories to receive additional options when creating an external table Nov 7, 2022
@timvw
timvw marked this pull request as ready for review November 7, 2022 14:39
@andygrove

Copy link
Copy Markdown
Member

@avantgardnerio fyi

@avantgardnerio

Copy link
Copy Markdown
Contributor

fyi

Thanks @andygrove . @timvw read my mind - this was actually underway in the early versions of the PR but I abandoned it.

LGTM = looks great to me :)

Comment thread datafusion/sql/src/parser.rs Resolved

// positive case: additional options (one entry) can be specified
let sql =
"CREATE EXTERNAL TABLE t STORED AS x OPTIONS ('k1' 'v1') LOCATION 'blahblah'";

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I expected to see 'k1' = 'v1' here. Is there a particular SQL dialect being emulated here?

@timvw timvw Nov 8, 2022 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The k1=v1 is what the code originally did...

After feedback from @avantgardnerio I changed to follow postgresql' create external table [ OPTIONS ( option 'value' [, ... ] ) ]

Comment thread datafusion/sql/src/parser.rs Resolved
@alamb
alamb merged commit 2a834d1 into apache:master Nov 8, 2022
@ursabot

ursabot commented Nov 8, 2022

Copy link
Copy Markdown

Benchmark runs are scheduled for baseline = e8fe042 and contender = 2a834d1. 2a834d1 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ec2-t3-xlarge-us-east-2] ec2-t3-xlarge-us-east-2
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on test-mac-arm] test-mac-arm
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ursa-i9-9960x] ursa-i9-9960x
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ursa-thinkcentre-m75q] ursa-thinkcentre-m75q
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate logical-expr Logical plan and expressions sql SQL Planner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow additional options when creating an external table

5 participants