Skip to content

[Epic] Data Driven Tests #4460

Closed
Closed

Description

@alamb

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
As described in #4248 and implemented in #4395 we now have a sqlogictest based test framework in DataFusion 🎉

This ticket tracks the various follow on work we would like to do

Describe the solution you'd like

Basically we would like to follow the DuckDB variant of sqllogictest as much as possible: https://duckdb.org/dev/sqllogictest/intro

The current rules for verification can be found in: https://duckdb.org/dev/sqllogictest/result_verification

Items

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
cc @mvanschellebeeck @xudong963

Activity

  1. xudong963 commented on Dec 1, 2022

    @xudong963
    Member

    I'll add more TODOs tomorrow.

  2. alamb commented on Dec 2, 2022

    @alamb
    ContributorAuthor

    @xudong963 I think a fun project might be to figure out how to support a minimal implementation of INSERT to insert data. Maybe by implementing something entirely in the sqllogictest harness using ArrayBuilders 🤔 It doesn't have to handle errors well initially or be performant.

    I wonder if something crazy like forming a single row record batch from the input, and then concat that batch with any existing batches would be good enough 🤔

  3. xudong963 commented on Dec 3, 2022

    @xudong963
    Member

    @alamb Today I have a more crazy idea: we can use Postgres to run create table and insert into ... for DF.

    To implement the idea, we can register two DBs for sqllogictest runner, PG and DF.

    For pg: we can register Postgres client in our logictest by use tokio_postgres::Client and start Postgres server in docker.

    Here is a draft PR to verify if my idea is feasible in sqllogictest-rs: xudong963/sqllogictest-rs#1

  4. alamb commented on Dec 3, 2022

    @alamb
    ContributorAuthor

    🤔 I wonder @xudong963 how will inserting data into postgres allow us to insert data into DataFusion 🤔

  5. alamb commented on Dec 3, 2022

    @alamb
    ContributorAuthor

    Also, heads up I have some ideas on how to improve the sqllogic framework #4492 (comment)

  6. xudong963 commented on Dec 3, 2022

    @xudong963
    Member

    🤔 I wonder @xudong963 how will inserting data into postgres allow us to insert data into DataFusion 🤔

    Seems it's impossible, my mistake.

  7. xudong963 commented on Dec 3, 2022

    @xudong963
    Member

    I wonder if something crazy like forming a single row record batch from the input, and then concat that batch with any existing batches would be good enough 🤔

    Make sense to me. But let me think more and find out if there are some other ways.

  8. mvanschellebeeck commented on Dec 4, 2022

    @mvanschellebeeck
    Contributor

    hey @alamb could you also add #4495 and #4494 as issues under the epic?

  9. xudong963 commented on Dec 4, 2022

    @xudong963
    Member

    hey @alamb could you also add #4495 and #4494 as issues under the epic?

    added

  10. alamb commented on Dec 4, 2022

    @alamb
    ContributorAuthor

    Update: check out the insert proposal from @xudong963 here #4496 🥳 🦜

  11. mvanschellebeeck commented on Dec 4, 2022

    @mvanschellebeeck
    Contributor

    @xudong963 / @alamb - could we also add #4502

  12. changed the title [-]Data Driven Tests [EPIC][/-] [+][Epic] Data Driven Tests[/+] on Mar 5, 2023
  13. liurenjie1024 commented on May 22, 2023

    @liurenjie1024
    Contributor

    @alamb Please help to add #6405 and assign it to me.

  14. alamb commented on May 22, 2023

    @alamb
    ContributorAuthor

    @alamb Please help to add #6405 and assign it to me.

    Done @liurenjie1024 -- thank you

  15. alamb commented on Jun 27, 2023

    @alamb
    ContributorAuthor

    I think I am going to claim this epic is substantially done so I am closing it out (we have pretty good data driven testing now) The rest of the stragglers I think are tracked in their own tickets without issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions