Skip to content

JsonStreamReaderTask / JsonStreamWriterTask: a scalar line raises a TypeError and the writer does not create the parent directory #207

Closed

Description

@njoubert-cleverage

Description

JsonStreamFile has two problems:

  1. Reader, scalar line: when a JSON Lines file contains a line that decodes to a scalar (42, "foo", true…), JsonStreamReaderTask crashes with a PHP TypeError coming from the ?array return type, without a useful message nor line number.
  2. Writer, missing directory: JsonStreamWriterTask fails when the parent directory of file_path does not exist, while CsvWriterTask creates it (see CsvFile).

Reproduction

clever_age_process:
    configurations:
        demo.file.json_stream_reader_scalar_line:
            tasks:
                content:
                    service: '@CleverAge\ProcessBundle\Task\ConstantOutputTask'
                    outputs: [ write ]
                    options:
                        output: "{\"a\":1}\n42\n"
                write:
                    service: '@CleverAge\ProcessBundle\Task\File\FileWriterTask'
                    outputs: [ read ]
                    options:
                        filename: '%kernel.project_dir%/var/data/json_stream_scalar_line.jsonl'
                read:
                    service: '@CleverAge\ProcessBundle\Task\File\JsonStream\JsonStreamReaderTask'
                    outputs: [ debug ]
                debug:
                    service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'

        demo.file.json_stream_writer_missing_dir:
            tasks:
                entry:
                    service: '@CleverAge\ProcessBundle\Task\ConstantIterableOutputTask'
                    outputs: [ write ]
                    options:
                        output:
                            - column1: value1-1
                            - column1: value1-2
                write:
                    service: '@CleverAge\ProcessBundle\Task\File\JsonStream\JsonStreamWriterTask'
                    outputs: [ debug ]
                    options:
                        file_path: '%kernel.project_dir%/var/data/json_stream_missing_dir/sub/out.jsonl'
                debug:
                    service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'
$ bin/console cleverage:process:execute demo.file.json_stream_reader_scalar_line
array:1 [
  "a" => 1
]
CRITICAL  [cleverage_process_task] CleverAge\ProcessBundle\Filesystem\JsonStreamFile::readLine(): Return value must be of type ?array, int returned

$ bin/console cleverage:process:execute demo.file.json_stream_writer_missing_dir
CRITICAL  [cleverage_process_task] SplFileObject::__construct(/var/www/var/data/json_stream_missing_dir/sub/out.jsonl): Failed to open stream: No such file or directory

Tested on main (c1a2804), PHP 8.5, Symfony 7.4.

Cause

Proposed fix

  • JsonStreamFile::readLine() checks the decoded value and throws an explicit \UnexpectedValueException naming the line number, the file path and the type found. The signature is unchanged and null lines are still skipped.
  • JsonStreamFile::__construct() creates the parent directory recursively, as CsvFile does, for non-read modes only (standard streams are skipped). JsonStreamWriterTask (and the --output option of cleverage:process:execute with the json-stream format) then work with a path whose directory does not exist yet.
  • Update the reference documentation, add unit tests.

Requirements

  • Documentation updates
    • Reference
    • Cookbooks
    • Changelog
  • Unit tests

Breaking changes

None. A scalar line now throws an \UnexpectedValueException instead of a \TypeError.

Activity

  1. added 2 commits that reference this issue on Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions