You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Expr::to_bytes can produce output that hits Expr::from_bytes recursion limit #3968
Describe the bug Expr::to_bytes with deeply nested expressions can produce a byte sequence that cannot be read by Expr::from_bytes because the prost recursion limit is hit. This is suboptimal in distributed application because it would be good if a client can decide if an expression is wire/roundtrip-safe and potentially adjust its predicate pushdown accordingly.
To Reproduce
#[test]fnroundtrip_deeply_nested(){// we need more stack space so this doesn't overflow in dev builds
std::thread::Builder::new().stack_size(10_000_000).spawn(|| {// don't know what "too much" is, so let's slowly try to increase complexitylet n_max = 100;for n in1..n_max {println!("testing: {n}");let expr_base = col("a").lt(lit(5i32));let expr = (0..n).fold(expr_base.clone(), |expr, _| expr.and(expr_base.clone()));// Convert it to an opaque formlet bytes = match expr.to_bytes(){Ok(bytes) => bytes,Err(_) => {// found expression that is too deeply nestedreturn;}};// Decode bytes from somewhere (over network, etc.let decoded_expr = Expr::from_bytes(&bytes).expect("serialization worked, so deserialization should work as well");assert_eq!(expr, decoded_expr);}panic!("did not find a 'too deeply nested' expression, tested up to a depth of {n_max}")}).expect("spawning thread").join().expect("joining thread");}
Expected behavior
Above test passes.
Additional context
Above test fails with:
thread '<unnamed>' panicked at 'serialization worked, so deserialization should work as well: Plan("Error decoding expr as protobuf: failed to decode Protobuf message: LogicalExprNode.expr_type: BinaryExprNode.l: LogicalExprNode.expr_type: .....recursion limit reached")'
Activity
fix: check recursion limit in `Expr::to_bytes`
fix: check recursion limit in `Expr::to_bytes` (#3970)
fix: check recursion limit in `Expr::to_bytes` (apache#3970)
fix: check recursion limit in `Expr::to_bytes` (apache#3970)