Repository navigation
Multiple Inputs & Outputs #30
Description
We've kind of gotten away with having a single input queue and output queue for each node.
However, this has led to some weird behavioural differences:
- For ProductPlugins, eg: Join, there must be exactly two inputs and they are consumed separately
- For everything else, there can be multiple inputs and they are consumed indiscriminately
This might not seem like much of a difference but it adds significant weirdness to the pipeline code.
Maybe it'd make sense for nodes to have 0, 1 or 2 input "ports" and have this visible in the UI.
Also, a lot of "filter" operations, eg: selecting records matching a pattern, could use a second output for
"failures". Effectively an "else" output. Sure you can add another complementary filter node but there's always
the risk that you'll miss something. Plus there's no output from writers. So maybe it'd make sense for nodes to have 0, 1 or 2 output "ports" and have this visible in the UI.
Activity
nickzoic commentedon Apr 2, 2024
plugins get
num_inputsandnum_outputsproperties in 9350584 and the GUI now prevents adding links where these are 0 but actually supporting multiple ports is a bigger piece of work ...nickzoic commentedon Oct 23, 2024
Note that the new(ish) Filter plugin is a step in this direction: currently the "outputs" option allows setting a column to a value, but it could also allow selection of between multiple output "ports".
afrubin commentedon Mar 12, 2026
@nickzoic I think this all makes sense to me and is worth doing but we should discuss how much work/how disruptive these changes are likely to be.