You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Transforming LogicalPlan::Explain using Treenode::transform fails unexpectedly #8396
In parseable we have external box for selecting timerange beside query. I want this external filter to be directly applied to any Table referenced in SQL so that this filter expression is available at scan time.
What is the bug ?
In the project I converted the SQL into LogicalPlan and used TreeNode::transform method to mutate the plan to include the external time range filter.
If the plan is Explain, transformation fails with error message. Otherwise it seems to work just fine
Plan("Invalid EXPLAIN command. Expression is empty")
To Reproduce
minimal program to reproduce this
use std::sync::Arc;use datafusion::{
arrow::{
array::BooleanArray,
datatypes::{DataType,Field,Fields,Schema},
record_batch::RecordBatch,},
common::tree_node::{Transformed,TreeNode},
datasource::{provider_as_source,MemTable},
logical_expr::{col,Expr,Filter,LogicalPlan,LogicalPlanBuilder},
scalar::ScalarValue,};fnmain(){// bit of boilerplate code let schema = Arc::new(Schema::new(Fields::from(vec![Field::new("a",DataType::Boolean,false,)])));let rb = RecordBatch::try_new(
schema.clone(),vec![Arc::new(BooleanArray::from(vec![true]))],).unwrap();let table = Arc::new(MemTable::try_new(schema,vec![vec![rb]]).unwrap());// example logical planlet plan = LogicalPlanBuilder::scan("table",provider_as_source(table),None).unwrap()// new explain is created here..explain(false,false).unwrap().build().unwrap();// example external filter that i want to addlet external_filter = col("a").eq(Expr::Literal(ScalarValue::Boolean(Some(true))));// after transformation, because plan is not the same anymore,// the parent plan is built again with call to LogicalPlan::with_new_inputs -> with_new_exprs
plan.transform(&|plan| match plan {LogicalPlan::TableScan(table) => {let filter = Filter::try_new(
external_filter.clone(),Arc::new(LogicalPlan::TableScan(table)),).unwrap();Ok(Transformed::Yes(LogicalPlan::Filter(filter)))}
x => Ok(Transformed::No(x)),})// This panics because LogicalPlan::with_new_exprs fail. Explain yeilds empty expression// Panic message - Invalid EXPLAIN command. Expression is empty.unwrap();// run the plan in datafusion session}
Activity
alamb commentedon Dec 1, 2023
Thank you for the report -- I agree explain plan may need special handling