Skip to content

Several tasks mishandle edge cases: AdvancedStatCounterTask off-by-one, IterableBatchTask rejects a null batch_count, ConditionTrait TypeError on scalars, ColumnAggregatorTask treats null as missing #206

Closed

Description

@njoubert-cleverage

Description

Four tasks or helpers fail on edge cases:

  1. AdvancedStatCounterTask tests counter % show_every before incrementing the counter: the first counted execution is never logged and the reported item count is one behind. With show_every: 1, 3 executions give: nothing, "1 items processed", "2 items processed".
  2. IterableBatchTask only accepts an integer batch_count: the null !== $batchCount branch ("only flush at the end", as in SimpleBatchTask) is dead code, and batch_count: ~ throws.
  3. ConditionTrait::checkValue(), checkEmpty() and getValue() are typed object|array: as soon as a condition is set, a scalar input raises a TypeError. The documented '' path, which targets the whole value, is therefore unusable on scalars, e.g. array_filter on a list of strings.
  4. ColumnAggregatorTask uses isset() to detect missing columns, so a column whose value is null counts as missing and throws an UnexpectedValueException.

Reproduction

clever_age_process:
    configurations:
        demo.tasks_edge.advanced_stat_counter:
            tasks:
                entry:
                    service: '@CleverAge\ProcessBundle\Task\ConstantIterableOutputTask'
                    options: { output: [a, b, c] }
                    outputs: [ stats ]
                stats:
                    service: '@CleverAge\ProcessBundle\Task\Reporting\AdvancedStatCounterTask'
                    options: { show_every: 1 }

        demo.tasks_edge.iterable_batch_null:
            tasks:
                entry:
                    service: '@CleverAge\ProcessBundle\Task\ConstantIterableOutputTask'
                    options: { output: [a, b, c] }
                    outputs: [ batch ]
                batch:
                    service: '@CleverAge\ProcessBundle\Task\IterableBatchTask'
                    options: { batch_count: ~ }
                    outputs: [ debug ]
                debug:
                    service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'

        demo.tasks_edge.condition_scalar:
            tasks:
                entry:
                    service: '@CleverAge\ProcessBundle\Task\ConstantOutputTask'
                    options: { output: [a, b] }
                    outputs: [ filter ]
                filter:
                    service: '@CleverAge\ProcessBundle\Task\TransformerTask'
                    options:
                        transformers:
                            array_filter:
                                condition: { match: { '': a } }
                    outputs: [ debug ]
                debug:
                    service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'

        demo.tasks_edge.column_aggregator_null:
            tasks:
                entry:
                    service: '@CleverAge\ProcessBundle\Task\ConstantOutputTask'
                    options: { output: { a: x, b: ~ } }
                    outputs: [ aggregate ]
                aggregate:
                    service: '@CleverAge\ProcessBundle\Task\ColumnAggregatorTask'
                    options: { columns: [a, b] }
                    outputs: [ debug ]
                debug:
                    service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'

Output on main (c1a2804, PHP 8.5, Symfony 7.4):

# advanced_stat_counter (-vv): 3 executions, only 2 logs, counts are off by one
INFO [cleverage_process_task] Last iteration 00:00:00 ago - n/a items/s - 1 items processed in 00:00:00
INFO [cleverage_process_task] Last iteration 00:00:00 ago - n/a items/s - 2 items processed in 00:00:00

# iterable_batch_null
CRITICAL [cleverage_process_task] The option "batch_count" with value null is expected to be of type "integer", but is of type "null".

# condition_scalar
CRITICAL [cleverage_process_task] Transformation 'array_filter' have failed: CleverAge\ProcessBundle\Transformer\Array\ArrayFilterTransformer::checkValue(): Argument #1 ($input) must be of type object|array, string given, called in .../src/Transformer/ConditionTrait.php on line 33

# column_aggregator_null
CRITICAL [cleverage_process_task] Missing columns [b] in input

Cause

Proposed fix

  • AdvancedStatCounterTask: increment the counter before the show_every test, so statistics are logged on the N-th, 2N-th… counted execution, with the right item count.
  • IterableBatchTask: allow ['integer', 'null'] for batch_count; null buffers every input and outputs them one by one on flush, like SimpleBatchTask.
  • ConditionTrait: type $input as mixed; the '' path targets the whole value, even a scalar; any other path on a scalar gives null, like a missing key.
  • ColumnAggregatorTask: detect missing columns with array_key_exists() for array inputs, so null columns are aggregated (ArrayAccess inputs keep the isset() check).
  • Update the reference documentation (AdvancedStatCounterTask, IterableBatchTask, ColumnAggregatorTask, FilterTask, ConditionTrait, ArrayFilterTransformer), add unit tests.

Requirements

  • Documentation updates
    • Reference
    • Cookbooks
    • Changelog
  • Unit tests

Breaking changes

None intended (bug fixes), but behaviour changes slightly:

  • AdvancedStatCounterTask now logs one execution earlier (on the N-th counted execution instead of the (N+1)-th), with correct item counts.
  • ColumnAggregatorTask now aggregates rows whose column value is null instead of throwing (or only warning with ignore_missing: true).
  • ConditionTrait methods now declare a mixed $input: a class overriding checkValue(), checkEmpty() or getValue() with the old object|array signature must widen it.

Activity

  1. added 2 commits that reference this issue on Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions