Skip to content

Incorrect answers with SELECT DISTINCT queries #250

Closed

Description

@alamb

Describe the bug
SELECT DISTINCT (note not distinct aggregates, eg SELECT COUNT (DISTINCT)...) produce incorrect results

To Reproduce

echo "A" > /tmp/foo.csv
echo "B" >> /tmp/foo.csv
echo "B" >> /tmp/foo.csv
echo "B" >> /tmp/foo.csv

Then in the datafusion-cli:

> CREATE EXTERNAL TABLE t(col varchar)
STORED AS CSV
LOCATION '/tmp/foo.csv';
0 rows in set. Query took 0 seconds.
> SELECT DISTINCT col from t
> ;
+-----+
| col |
+-----+
| A   |
| B   |
| B   |
| B   |
+-----+

Expected behavior
Expected results are no duplicates. The output should contain only 2 rows with values of A and B

Additional context
I suspect we are just ignoring the distinct on these queries. We could "fix" the bug by generating an error about the feature being unsupported. Bonus points would be to add a group by operator on top of the plan to remove duplicates (e.g. group by all the items in the select list)

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions