You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
hash_join panics when join keys have different data types #2877
There are several places where hash_join can panic in version 9.0.0 when I run SQL queries.
thread 'tokio-runtime-worker' panicked at 'called `Option::unwrap()` on a `None` value', /home/andy/.cargo/registry/src/github.com-1ecc6299db9ec823/datafusion-9.0.0/src/physical_plan/hash_join.rs:973:17
thread 'tokio-runtime-worker' panicked at 'called `Option::unwrap()` on a `None` value', /home/andy/.cargo/registry/src/github.com-1ecc6299db9ec823/datafusion-9.0.0/src/physical_plan/hash_join.rs:979:17
thread 'tokio-runtime-worker' panicked at 'called `Option::unwrap()` on a `None` value', /home/andy/.cargo/registry/src/github.com-1ecc6299db9ec823/datafusion-9.0.0/src/physical_plan/hash_join.rs:1045:17
To Reproduce
I am experimenting with SQL query fuzzing. Here is one query that failed:
SELECT _c438, _c439, _c440, _c441, _c442, _c443
FROM (
(
(SELECTtest1.c0AS _c438, test1.c1AS _c439, test1.c2AS _c440, test1.c3AS _c441, test1.c4AS _c442, test1.c5AS _c443
FROM (test1))
FULL JOIN
(SELECTtest1.c0AS _c444, test1.c1AS _c445, test1.c2AS _c446, test1.c3AS _c447, test1.c4AS _c448, test1.c5AS _c449
FROM (test1))
ON _c438 = _c446)
RIGHT JOIN
(SELECTtest0.c0AS _c450, test0.c1AS _c451, test0.c2AS _c452, test0.c3AS _c453, test0.c4AS _c454, test0.c5AS _c455
FROM (test0))
ON _c441 = _c451);
I think the issue is that we don't have type coercion for join keys so joining a number column to a string can cause this. in fn equal_rows we are making assumptions that left and right columns have the same data type.
changed the title [-]hash_join panics ('called `Option::unwrap()` on a `None` value)[/-][+]hash_join panics when join keys have different data types[/+]on Jul 12, 2022
Activity
andygrove commentedon Jul 12, 2022
I think the issue is that we don't have type coercion for join keys so joining a number column to a string can cause this. in
fn equal_rowswe are making assumptions that left and right columns have the same data type.[-]hash_join panics ('called `Option::unwrap()` on a `None` value)[/-][+]hash_join panics when join keys have different data types[/+]comphead commentedon Jul 18, 2022
@andygrove I'd like to investigate this as part of #2910
comphead commentedon Aug 2, 2022
Rev engineered data from sqlfuzz, quick test to reproduce the behavior
Right, the problem is non compat downcast