Skip to content

[Rust] [DataFusion] Add join support to DataFrame and LogicalPlan #26548

Closed

Description

@asfimport

Add join support to DataFrame and LogicalPlan.

Logical Plan

My initial thoughts on the design of the LogicalPlan struct would be:

struct InnerJoin {
  left: Box<LogicalPlan>,
  right: Box<LogicalPlan>,
  left_keys: Vec<Expr>,
  right_keys: Vec<Expr>
} 

The left_keys and right_keys vectors must have the same length. Example pseudo-code:

let join = InnerJoin {
  left: read_parquet("customers"),
  right: read_parquer("orders"),
  left_keys: vec![col("id")],
  right_keys: vec![col("customer_id")]
};    

DataFrame

let customer = ctx.read_parquet("customers").alias("c");
let orders = ctx.read_parquet("orders").alias("o");

// generic join method that can support all types of join
let join = customer.join(orders, col("c.id").eq("o.customer_id"))

// or we could start with a more specific equijoin method
let join = customer.inner_join(orders, vec![col("id")], vec![col("customer_id")]);

 

 

 

 

Reporter: Andy Grove / @andygrove
Assignee: Andy Grove / @andygrove

PRs and other links:

Note: This issue was originally created as ARROW-10585. Please see the migration documentation for further details.

Activity

  1. asfimport commented on Nov 21, 2020

    @asfimport
    Author

    Andy Grove / @andygrove:
    Issue resolved by pull request 8720
    #8720

  2. added this to the 3.0.0 milestone on Jan 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions