You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Should not fold a non-constant expression when it is not clear to do so #1244
Describe the bug
This expression col < 10 and boolean(NULL) can either be NULL or false depending on the result of col < 10. Hence it should not be rewritten to a constant but it is current can.
To Reproduce
Add the following test in optimize_expr_bool_and, you will see it was rewritten to c1 BETWEEN Int32(0) AND Int32(10) which is wrong.
// c1 BETWEEN Int32(0) AND Int32(10) AND Boolean(NULL)
// it can be either NULL or false depending on the value of `c1 BETWEEN Int32(0) AND Int32(10`
let expr = Expr::Between{ expr: Box::new(col("c1")), negated: false, low: Box::new(lit(0)), high: Box::new(lit(10))};
let expr = expr.and(Expr::Literal(ScalarValue::Boolean(None)));
let result = expr.clone().rewrite(&mut rewriter)?;
println!("expr: {:#?} \n result: {:#?}", expr, result);
assert_eq!(expr, result);
Expected behavior
Do not rewritten expression that does not have deterministic result yet
changed the title [-]Should not fold a non-constant expression[/-][+]Should not fold a non-constant expression when it is not clear to do so[/+]on Nov 4, 2021
👍 FWIW I think handling NULLs correctly is the single hardest thing in databases (at least the thing that is the most fraught with strange corner cases). The fact we are dealing with them in DataFusion now I think is a sign of its maturity
Activity
[-]Should not fold a non-constant expression[/-][+]Should not fold a non-constant expression when it is not clear to do so[/+]NGA-TRAN commentedon Nov 4, 2021
I am working on a fix for this @alamb
alamb commentedon Nov 4, 2021
thank you @NGA-TRAN -- FYI @matthewmturner I lead you astray in #1208 🤦 about nulls. NULLs!!!!
matthewmturner commentedon Nov 5, 2021
@alamb got it - thanks for keeping me posted!
alamb commentedon Nov 6, 2021
👍 FWIW I think handling
NULLs correctly is the single hardest thing in databases (at least the thing that is the most fraught with strange corner cases). The fact we are dealing with them in DataFusion now I think is a sign of its maturity